[GitHub] bigtop pull request #361: BIGTOP-2886: Update solr smoke test to work with n...

2018-05-27 Thread JunHe77
GitHub user JunHe77 opened a pull request:

https://github.com/apache/bigtop/pull/361

BIGTOP-2886: Update solr smoke test to work with new 6.6.0 version

Change-Id: Ib549231e3205bbd26881ce115e34d03018ff9b1c
Signed-off-by: Jun He 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JunHe77/bigtop BIGTOP-2886

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/bigtop/pull/361.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #361


commit 8769dd8aaefb8cad9b16a0439dc5af9865f038c2
Author: Jun He 
Date:   2017-11-09T09:05:50Z

BIGTOP-2886: Update solr smoke test to work with new 6.6.0 version

Change-Id: Ib549231e3205bbd26881ce115e34d03018ff9b1c
Signed-off-by: Jun He 




---


Re: Create multiarch docker images for bigtop-puppet/slaves

2018-05-27 Thread Jun HE
Thanks for comments, Evans.

There should not be much efforts to enable multi-arch. The question is how
we plan to support it in Bigtop.

I can think of several items to discuss for this feature:
1. Docker images name conventions:
For this, I'd suggest to using this:
* for actual underlying docker image running on specific platform, name
it as:
bigtop/:---,
say: bigtop/puppet:trunk-debian-9-amd64,
bigtop/slaves:1.3.0-fedora-26-aarch64
* for virtual image, name it as: bigtop/:
--,
say:  bigtop/puppet:trunk-debian-9,  bigtop/slaves:1.3.0-fedora-26
2. How to create these images/manifests
I didn't find any cmds/scripts in Bigtop source to do the docker hub
uploading work. So I guess we're doing it manually for now. If we're going
to use multi-arch, we need to upload the virtual image's manifest file to
docker hub besides those arch dependent images.
Docker cli has a cmd to do such work: docker manifest. But it is not in
distros' docker releases. And it is marked as "experimental" feature, user
needs to change docker settings to use it. See
https://docs.docker.com/edge/engine/reference/commandline/manifest/#description.
I created a shell script version (
https://github.com/JunHe77/docker-manifest-sh) to do the same thing, use
same cmd syntax. Hope this can help.
3. Update existing settings/yaml files to use the virutal distro image.
Once the images/manifests are ready in docker hub, we can update
existing settings/yaml files to use "virtual" bigtop images. This should
take too much efforts. And we'll get unified, beautiful code for different
archs.

Pls raise your thoughts, questions, and etc. Once we settle these
workflows/specs down, I'm sure it will be ready before Sept.

Thanks.

2018-05-28 1:45 GMT+08:00 Evans Ye :

> I'd say this is a great idea.
> The only reason we haven't adopt multi-arch is because we started to adopt
> docker at early stage so there's no such feature then.
>
> Jun can you roughly estimate the effort? If we're going for it, we better
> have it ready in 1.3.0 before ApacheCon which is around September.
>
>
> 2018-05-17 17:19 GMT+08:00 Naresh Bhat :
>
> > On 17 May 2018 at 13:21, Jun HE  wrote:
> > > Hi, guys,
> > >
> > > As bigtop has simplified the way to create docker images for distros,
> by
> > > BIGTOP-2922, I'm thinking of multi-arch docker images support to ease
> > > building/test/development.
> > >
> > > Multi-arch feature has been supported since 2017, you may check this
> > link:
> > > https://blog.docker.com/2017/11/multi-arch-all-the-things/
> > > In simple words, multi-arch is a virtual image which contains several
> > > architecture dependent images. When user try to pull this virtual image
> > > dockhub will send back the actual image based on the architecture he is
> > > using.
> > >
> > > Use multi-arch will help:
> > > * development: user don't have to issue different cmd to pull
> > puppet/slaves
> > > images, "docker pull bigtop/slaves:trunk-debian-9" is enough instead of
> > "docker
> > > pull bigtop/slaves:trunk-debian-9-ppc64le" or "docker pull
> > > bigtop/slaves:trunk-debian-9-aarch64"
> > > * test: unified yaml file for same distro. refer example in
> > > provisioner/docker/config_*
> > >
> > > So how do you think? :)
> >
> > Thanks Jun for initiating the thread.
> > +1 this makes sense. Is there any reason why we have not consider for
> > using multi-arch images ? If anybody explains advantage/disadvantage
> > of using multi-arch docker image in bigtop, it will help us to
> > understand in a better way.
> >
> > >
> > > Regards,
> > >
> > > Jun
> >
>


Re: Create multiarch docker images for bigtop-puppet/slaves

2018-05-27 Thread Evans Ye
I'd say this is a great idea.
The only reason we haven't adopt multi-arch is because we started to adopt
docker at early stage so there's no such feature then.

Jun can you roughly estimate the effort? If we're going for it, we better
have it ready in 1.3.0 before ApacheCon which is around September.


2018-05-17 17:19 GMT+08:00 Naresh Bhat :

> On 17 May 2018 at 13:21, Jun HE  wrote:
> > Hi, guys,
> >
> > As bigtop has simplified the way to create docker images for distros, by
> > BIGTOP-2922, I'm thinking of multi-arch docker images support to ease
> > building/test/development.
> >
> > Multi-arch feature has been supported since 2017, you may check this
> link:
> > https://blog.docker.com/2017/11/multi-arch-all-the-things/
> > In simple words, multi-arch is a virtual image which contains several
> > architecture dependent images. When user try to pull this virtual image
> > dockhub will send back the actual image based on the architecture he is
> > using.
> >
> > Use multi-arch will help:
> > * development: user don't have to issue different cmd to pull
> puppet/slaves
> > images, "docker pull bigtop/slaves:trunk-debian-9" is enough instead of
> "docker
> > pull bigtop/slaves:trunk-debian-9-ppc64le" or "docker pull
> > bigtop/slaves:trunk-debian-9-aarch64"
> > * test: unified yaml file for same distro. refer example in
> > provisioner/docker/config_*
> >
> > So how do you think? :)
>
> Thanks Jun for initiating the thread.
> +1 this makes sense. Is there any reason why we have not consider for
> using multi-arch images ? If anybody explains advantage/disadvantage
> of using multi-arch docker image in bigtop, it will help us to
> understand in a better way.
>
> >
> > Regards,
> >
> > Jun
>