Bug#948211: Decouple containerd from docker.io again?

[Shengjing Zhu]

On Tue, Jan 7, 2020 at 11:22 AM Arnaud Rebillout
 wrote:
[..]
> 1) try to update containerd -> oh, requires a new version of containerd/cri
> 2) so let's try to update containerd/cri -> oh, requires a new version of 
> docker

The plan is to keep upstream
vendor/github.com/{docker/docker,containerd/cri} in src:containerd.
Since the vendor/github.com/docker/docker/ only has ~60 Go files, it's
easy to review copyright and other things.

>
> I think you can already upload containerd to unstable at this point? Or is 
> there some blocker?

There's no blocker, but I want to ensure that the docker.io
maintainers are aware of it.

-- 
Shengjing Zhu

Bug#948211: Decouple containerd from docker.io again?

[Arnaud Rebillout]

On 1/7/20 1:29 AM, Shengjing Zhu wrote:

We should keep containerd version in sync with docker. eg docker
19.03.x with containerd 1.2.x. And I guest next docker major version
will use containerd 1.3.x.



Agreed on that, as long as both projects are in sync upstream, that can 
work.






Talking about versions, looking at the file 
https://download.docker.com/linux/debian/dists/buster/stable/binary-amd64/Packages,
 I can see that:

- the docker-ce package started to depend on container.io from the 18.09.x 
series (it's been a while: good)
- the latest version of docker-ce "5:19.03.5~3-0~debian-buster" depends on 
containerd.io (>= 1.2.2-3)... Where is the source for this package? Is containerd.io 
patched here?

I found a comment from someone asking for the sources of the package: 
https://github.com/containerd/containerd/issues/1508#issuecomment-461413010. 
The comment went unanswered. A bit more research, and still no idea where are 
the source for this package. Do you have an idea?


Neither do I :(

But let's be optimized about this. There's no reason for docker
upstream to _hide_ some patches which are not applied in containerd
upstream :)
Actually if you look at commits in containerd release branch, the
people from docker are actively backporting things.



I don't think anyone hides anything, but I don't want to be in the 
situation were:


- you install docker and containerd from docker repos, everything works
- you install docker and containerd from debian repos (same versions as 
in docker repos), and some things don't work...


Why? Because there are some patches in containerd.io (from docker repos) 
that we are not aware of...


That's one thing I'm worried about as long as I don't have the source 
package for containerd.io.


The second thing is that, in any case, we shouldn't patch containerd to 
accomodate docker. If docker is the consumer of containerd, and can't 
work with it "as is", then docker has to be patched, not containerd.


All of that is theoretical speaking, maybe in the end docker and 
containerd can work with each other, from tagged version, and without 
any patch.


But this has never been the case up to now (I mean, docker could never 
/build/ against a tagged version of containerd, at least), so I have 
reasons to be cautious, and not over optimistic.





CRI makes containerd useful without docker in a kubernetes worker
node. (That's why I want a separate containerd package)

I have some mistakes in my previous sentence. It should be,
+ github.com/containerd/cri pull in github.com/docker/docker.
+ github.com/containerd/containerd/cmd/containerd pull in
github.com/containerd/cri
So only when you build containerd binary(with cri), you will need
github.com/docker/docker.
No consumer will pull in github.com/containerd/containerd/cmd/..., so
no need to pull in github.com/docker/docker.
There's no circular dependencies, apparently.



Thanks for the details.

Correct me if I'm wrong, but there's still a bit of a circular dep, in 
the sense that the Build-Depends are for the source package, not for 
each binary package produced.


So, assuming that we want to bump both docker and containerd to a new 
major version, the situation could very well be:


1) try to update containerd -> oh, requires a new version of containerd/cri
2) so let's try to update containerd/cri -> oh, requires a new version 
of docker

3) so let's try to update docker -> oh, requires a new version of containerd
4) back to 1...

That wouldn't be the first circular of this kind, we already had it at 
some point with fsouza-go-dockerclient, even though I think these days 
it's gone.


Anyway.

I think you can already upload containerd to unstable at this point? Or 
is there some blocker?


I'm not fundamentally against attempts to un-bundle containerd from 
docker, but the reality is that I don't know when (if ever) I can spend 
time giving it a try, and in the meantime you shouldn't be waiting on 
me. The docker package shouldn't block the containerd package from being 
in unstable, and unless I'm mistaken, at the moment it doesn't.


In the short-term, for unstable, I would be in favor of having 
containerd, and also docker.io as it is now, ie. with bundled 
containerd. And in a longer term, possibly before bullseyes, if it seems 
realistic, I will find some time to work on un-bundling.


Having someone working on the containerd packaging is already a great 
improvement, so it's good that this discussion is taking place.


Thanks!

  Arnaud

Bug#948211: Decouple containerd from docker.io again?

[Shengjing Zhu]

On Mon, Jan 6, 2020 at 3:02 PM Arnaud Rebillout
 wrote:
[...]
> Thanks for the details. So if I understand properly, the situation for 
> upstream docker is that:
>
> - the build dependency is against a containerd commit from the master branch 
> (pulled in according to the vendor.conf file)
> - however for runtime, docker-ce depends on an external containerd.io 
> package. The version of this package suggests that it's a released version 
> and not a snapshot.
>
> Do you understand the same?
>

Yes.

> Maybe we could try a similar middle ground for our packaging, ie. build with 
> the vendored copy of containerd, but don't ship the containerd binary, and 
> instead depend on the containerd package.
>
> But if we want this to work, I guess we should stick to the versions that are 
> tested by docker upstream, which might be an issue (if you want to track 
> latest containerd and docker lags behind, what do we do?).
>

We should keep containerd version in sync with docker. eg docker
19.03.x with containerd 1.2.x. And I guest next docker major version
will use containerd 1.3.x.

> Talking about versions, looking at the file 
> https://download.docker.com/linux/debian/dists/buster/stable/binary-amd64/Packages,
>  I can see that:
>
> - the docker-ce package started to depend on container.io from the 18.09.x 
> series (it's been a while: good)
> - the latest version of docker-ce "5:19.03.5~3-0~debian-buster" depends on 
> containerd.io (>= 1.2.2-3)... Where is the source for this package? Is 
> containerd.io patched here?
>
> I found a comment from someone asking for the sources of the package: 
> https://github.com/containerd/containerd/issues/1508#issuecomment-461413010. 
> The comment went unanswered. A bit more research, and still no idea where are 
> the source for this package. Do you have an idea?
>

Neither do I :(

But let's be optimized about this. There's no reason for docker
upstream to _hide_ some patches which are not applied in containerd
upstream :)
Actually if you look at commits in containerd release branch, the
people from docker are actively backporting things.

> That sounds like a red flag if we don't know what patches (if any) are 
> applied to the containerd.io package shipped by docker in their repo.
>
>
> containerd(without cri) don't import github.com/docker/docker.
> But containerd-cri needs some functions from docker repo.
> But you can build containerd with no_cri tag(I think this should be
> used when building docker, next release).
>
>
> Ok, good to know. Does that mean that you would build containerd with 
> `no_cri`, so that the containerd-dev package would not depend on docker-dev? 
> (I have no idea what CRI really does and if it's needed in Debian).
>

CRI makes containerd useful without docker in a kubernetes worker
node. (That's why I want a separate containerd package)

I have some mistakes in my previous sentence. It should be,
+ github.com/containerd/cri pull in github.com/docker/docker.
+ github.com/containerd/containerd/cmd/containerd pull in
github.com/containerd/cri
So only when you build containerd binary(with cri), you will need
github.com/docker/docker.
No consumer will pull in github.com/containerd/containerd/cmd/..., so
no need to pull in github.com/docker/docker.
There's no circular dependencies, apparently.

-- 
Shengjing Zhu

Bug#948211: Decouple containerd from docker.io again?

[Arnaud Rebillout]

On 1/6/20 12:41 PM, Shengjing Zhu wrote:

I think I can work with upstream (at least) to use containerd
release/1.3 branch for next release.



That would be awesome!



But I think generally upstream has managed to use released containerd
version, at least in their deb/rpm repo.
(IIRC they have built containerd separately last year or so.)
https://download.docker.com/linux/debian/dists/buster/pool/stable/amd64/
You can see containerd-1.2.10 things.



Thanks for the details. So if I understand properly, the situation for 
upstream docker is that:


- the *build dependency *is against a containerd commit from the master 
branch (pulled in according to the vendor.conf file)
- however for *runtime*, docker-ce depends on an external containerd.io 
package. The version of this package /suggests/ that it's a released 
version and not a snapshot.


Do you understand the same?

Maybe we could try a similar middle ground for our packaging, ie. build 
with the vendored copy of containerd, but don't ship the containerd 
binary, and instead depend on the containerd package.


But if we want this to work, I guess we should stick to the versions 
that are tested by docker upstream, which might be an issue (if you want 
to track latest containerd and docker lags behind, what do we do?).


Talking about versions, looking at the file 
https://download.docker.com/linux/debian/dists/buster/stable/binary-amd64/Packages, 
I can see that:


- the docker-ce package started to depend on container.io from the 
18.09.x series (it's been a while: good)
- the latest version of docker-ce "5:19.03.5~3-0~debian-buster" depends 
on containerd.io (>= 1.2.2-3)... Where is the source for this package? 
Is containerd.io patched here?


I found a comment from someone asking for the sources of the package: 
https://github.com/containerd/containerd/issues/1508#issuecomment-461413010. 
The comment went unanswered. A bit more research, and still no idea 
where are the source for this package. Do you have an idea?


That sounds like a red flag if we don't know what patches (if any) are 
applied to the containerd.io package shipped by docker in their repo.




containerd(without cri) don't import github.com/docker/docker.
But containerd-cri needs some functions from docker repo.
But you can build containerd with no_cri tag(I think this should be
used when building docker, next release).



Ok, good to know. Does that mean that you would build containerd with 
`no_cri`, so that the containerd-dev package would not depend on 
docker-dev? (I have no idea what CRI really does and if it's needed in 
Debian).



Cheers,

Bug#948211: Decouple containerd from docker.io again?

[Shengjing Zhu]

On Mon, Jan 6, 2020 at 11:31 AM Arnaud Rebillout
 wrote:
>
>Hi Shengjing,
>
>
>  > I think nowadays the docker upstream has much improvement on the
> dependencies versions.
>
> there are two things that prevent the docker.io package to depend on the
> containerd package.
>
>
>  1st issue: versions 
>
> On the branch `19.03`, docker vendors containerd as such:
>
>github.com/containerd/containerd 7c1e88399ec0b0b077121d9d5ad97e647b11c870
> https://github.com/containerd/containerd/commit/7c1e88399ec0b0b077121d9d5ad97e647b11c870
>
>
> which is somewhere on the master branch (date: 08-05-2019), and after
> the tag v1.2.0.
>
> On the branch `master`, docker vendors containers as such:
>
>github.com/containerd/containerd acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b
> https://github.com/containerd/containerd/commit/acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b
>
> which is somewhere on the master branch (date: 14-10-2019), and after
> the tag v1.3.0.
>
> Docker never tried to settled on a tag of containerd, and I see that to
> this date it's still true: docker will just vendor whatever containerd
> commit from the master branch.
>
> For Debian packaging it's obviously an issue, because you will package a
> specific tag of containerd, and then it's likely docker won't build
> against it, and we'll have to do some desperate patching to make the
> pieces fit together (last time I looked into it, the patching involved
> was not trivial).
>

Oh, there's still some mess.

I think I can work with upstream (at least) to use containerd
release/1.3 branch for next release.

But I think generally upstream has managed to use released containerd
version, at least in their deb/rpm repo.
(IIRC they have built containerd separately last year or so.)
https://download.docker.com/linux/debian/dists/buster/pool/stable/amd64/
You can see containerd-1.2.10 things.

>
>  2nd issue: circular dependencies 
>
> I'm not sure how much it's still true, but the last time I looked into
> it, there was some non-trivial circular dependencies between both
> packages, making it impractical, or impossible, to package both separately.
>
> Do you know if containerd still build depends on Docker these days? If
> so, how much of docker codebase is pulled in?
>
>

containerd(without cri) don't import github.com/docker/docker.
But containerd-cri needs some functions from docker repo.
But you can build containerd with no_cri tag(I think this should be
used when building docker, next release).

In the src:containerd/experimental, I kept docker/docker in vendor to
avoid circular problem.
https://salsa.debian.org/go-team/packages/containerd/tree/master/vendor/github.com/docker

-- 
Shengjing Zhu

Bug#948211: Decouple containerd from docker.io again?

[Arnaud Rebillout]

  Hi Shengjing,


> I think nowadays the docker upstream has much improvement on the 
dependencies versions.


there are two things that prevent the docker.io package to depend on the 
containerd package.



 1st issue: versions 

On the branch `19.03`, docker vendors containerd as such:

  github.com/containerd/containerd 7c1e88399ec0b0b077121d9d5ad97e647b11c870
https://github.com/containerd/containerd/commit/7c1e88399ec0b0b077121d9d5ad97e647b11c870 



which is somewhere on the master branch (date: 08-05-2019), and after 
the tag v1.2.0.


On the branch `master`, docker vendors containers as such:

  github.com/containerd/containerd acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b
https://github.com/containerd/containerd/commit/acdcf13d5eaf0dfe0eaeabe7194a82535549bc2b

which is somewhere on the master branch (date: 14-10-2019), and after 
the tag v1.3.0.


Docker never tried to settled on a tag of containerd, and I see that to 
this date it's still true: docker will just vendor whatever containerd 
commit from the master branch.


For Debian packaging it's obviously an issue, because you will package a 
specific tag of containerd, and then it's likely docker won't build 
against it, and we'll have to do some desperate patching to make the 
pieces fit together (last time I looked into it, the patching involved 
was not trivial).



 2nd issue: circular dependencies 

I'm not sure how much it's still true, but the last time I looked into 
it, there was some non-trivial circular dependencies between both 
packages, making it impractical, or impossible, to package both separately.


Do you know if containerd still build depends on Docker these days? If 
so, how much of docker codebase is pulled in?



> So if you like the option a, I could help move the current 
src:containerd from experimental to unstable


Is there anything that blocks you here? I indeed like option a: if 
there's a containerd package in debian, it should be build from 
src:containerd. The fact that docker embeds its own copy shouldn't 
prevent you from providing a containerd package.


I'm not quite sure that we nailed down the Replaces/Breaks/Conflicts 
that are needed though, we might need to work a bit on that.


At the moment, the docker package installs containerd as 
`docker-containerd`, and uses this binary, so I *think* that both 
containerd and docker could be installed in parallel. That's why there's 
no Breaks/Conflicts for those packages. I don't think this was tested 
though.


We also have this in the docker.io package:

  Replaces: golang-github-containerd-containerd-dev
  Breaks: golang-github-containerd-containerd-dev
  Provides: golang-github-containerd-containerd-dev

So basically, it means that a package that Build-Depends on 
golang-github-docker-docker-dev will pull in docker's version of 
containerd-dev.



All in all, the idea is that docker.io shouldn't block anyone from 
packaging containerd. I wish that the docker.io package could depend on 
containerd, instead of vendoring it. But as long as the two main issues 
mentioned above are not solved upstream, there's little chance we'll 
solve it in Debian.


Cheers,

  Arnaud



On 1/5/20 7:28 PM, Shengjing Zhu wrote:

Source: docker.io
Severity: wishlist

Hi Dmitry Smirnov and Arnaud Rebillout,

I would like to have a standalone containerd package. Containerd has
CRI(Kubernetes Container Runtime Interface) support[1].
This makes it possible to run containerd alone without docker, eg on a
Kubernetes node[2]. And I don't need features like `docker build` in
this scenario.

Would you like to:
a. build a separate containerd package from src:containerd
b. build a separate containerd package from src:docker.io

I think nowadays the docker upstream has much improvement on the
dependencies versions. So a separate src:containerd makes sense. I
checked the docker-ce repo, it has pull containerd 1.3 to its master.
So if you like the option a, I could help move the current
src:containerd from experimental to unstable, when the next docker
major release happens. (of course, the version of containerd should
sync with the docker release.)

[1] https://github.com/containerd/cri/
[2] 
https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/

Bug#948211: Decouple containerd from docker.io again?

[Shengjing Zhu]

Source: docker.io
Severity: wishlist

Hi Dmitry Smirnov and Arnaud Rebillout,

I would like to have a standalone containerd package. Containerd has
CRI(Kubernetes Container Runtime Interface) support[1].
This makes it possible to run containerd alone without docker, eg on a
Kubernetes node[2]. And I don't need features like `docker build` in
this scenario.

Would you like to:
a. build a separate containerd package from src:containerd
b. build a separate containerd package from src:docker.io

I think nowadays the docker upstream has much improvement on the
dependencies versions. So a separate src:containerd makes sense. I
checked the docker-ce repo, it has pull containerd 1.3 to its master.
So if you like the option a, I could help move the current
src:containerd from experimental to unstable, when the next docker
major release happens. (of course, the version of containerd should
sync with the docker release.)

[1] https://github.com/containerd/cri/
[2] 
https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/

-- 
Shengjing Zhu