Re: [Piglit] [PATCH v4] travis: add docker based cmake build job

2018-02-06 Thread Andres Gomez 
Thanks, Dylan, Eric and Juan!

I've just pushed this changed. ☺

PS: In my Travis-CI, the job is taking ~13 min. right now.

On Mon, 2018-02-05 at 09:29 -0800, Dylan Baker wrote:
> I haven't looked too closely at this, but I like the idea of using docker to
> allow running the cmake based job.
> 
> Acked-by: Dylan Baker 
> 
> Quoting Andres Gomez (2018-01-31 13:11:52)
> > Until now we were only running the python unit tests.
> > 
> > It seems desirable to also check that the CMake based build compiles
> > successfully. We do that now using docker.
> > 
> > The docker build can be tweaked with some environment variables and,
> > also, be stored in the docker hub if desired. Check the changes for
> > extra details regarding these variables.
> > 
> > v2: Removed other build possibilities other than just from inside
> > Travis-CI, as suggested by Juan.
> > v3: Replaced the "RELEASE" parameter to create the docker image with
> > "PARENT" and removed some unneeded documentation after v2, as
> > suggested by Juan.
> > v4:
> >   - Use DOCKER_PARENT, DOCKER_IMAGE and DOCKER_TAG variables from the
> > custom Travis-CI setup to define the PARENT, IMAGE and TAG
> > parameters during the docker image creation.
> >   - Upload the image only if DOCKER_IMAGE and DOCKER_TAG are set.
> > 
> > Cc: Dylan Baker 
> > Cc: Juan A. Suarez 
> > Signed-off-by: Andres Gomez 
> > Reviewed-by: Juan A. Suarez 
> > ---
> >  .travis.yml  | 47 --
> >  docker/Rockerfile.piglit | 74 
> > 
> >  2 files changed, 118 insertions(+), 3 deletions(-)
> >  create mode 100644 docker/Rockerfile.piglit
> > 
> > diff --git a/.travis.yml b/.travis.yml
> > index b47829ff9..b4cbcca49 100644
> > --- a/.travis.yml
> > +++ b/.travis.yml
> > @@ -1,5 +1,16 @@
> > +sudo: false
> > +os: linux
> >  language: python
> > -cache: pip
> > +cache:
> > +  - ccache
> > +  - pip
> > +services:
> > +  - docker
> > +
> > +env:
> > +  global:
> > +- BUILD=pytest
> > +
> >  matrix:
> >include:
> >  - python: 2.7
> > @@ -12,7 +23,37 @@ matrix:
> >env: TOX_ENV="py35-{generator,noaccel,accel-nix,streams}"
> >  - python: 3.6
> >env: TOX_ENV="py36-{generator,noaccel,accel-nix,streams}"
> > +- env: BUILD=cmake
> > +
> >  install:
> > -pip install tox
> > +  - |
> > +if [[ $BUILD == pytest ]]; then
> > +  pip install tox
> > +else
> > +  wget 
> > https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz
> > +  tar xvf rocker-1.3.1-linux_amd64.tar.gz
> > +  rm rocker-1.3.1-linux_amd64.tar.gz
> > +fi
> > +
> > +before_script:
> > +  - |
> > +if [[ $BUILD != pytest ]]; then
> > +  mkdir -p -m777 ~/.ccache
> > +fi
> > +
> >  script:
> > -- tox -e $TOX_ENV
> > +  - |
> > +if [[ $BUILD == pytest ]]; then
> > +  tox -e $TOX_ENV
> > +else
> > +  ./rocker build -f docker/Rockerfile.piglit ${DOCKER_TAG:+--var 
> > TAG=}${DOCKER_TAG} ${DOCKER_IMAGE:+--var IMAGE=}${DOCKER_IMAGE} 
> > ${DOCKER_PARENT:+--var PARENT=}${DOCKER_PARENT} .
> > +fi
> > +
> > +after_success:
> > +  - |
> > +if [[ $BUILD != pytest ]]; then
> > +  if [[ -n $DOCKER_IMAGE && -n $DOCKER_TAG && -n $DOCKER_USERNAME && 
> > $TRAVIS_BRANCH == master ]]; then
> > +docker login -u "${DOCKER_USERNAME}" -p "${DOCKER_PASSWORD}"
> > +docker push "${DOCKER_IMAGE}":"${DOCKER_TAG}"
> > +  fi
> > +fi
> > diff --git a/docker/Rockerfile.piglit b/docker/Rockerfile.piglit
> > new file mode 100644
> > index 0..0d8f8bb45
> > --- /dev/null
> > +++ b/docker/Rockerfile.piglit
> > @@ -0,0 +1,74 @@
> > +#
> > +# This builds Piglit.
> > +#
> > +# ~~~
> > +#  rocker build -f Rockerfile.piglit [--attach]
> >  \
> > +#[--var IMAGE=freedesktop/mesa] # freedesktop/mesa, myrepo/myproyect, 
> > ...   \
> > +#[--var TAG=piglit] # piglit-17.0, piglit-13.0, ...
> >  \
> > +#[--var PARENT=ubuntu:xenial]   # ubuntu:xenial, 
> > freedesktop/mesa:17.3, ...
> > +# ~~~
> > +#
> > +# Environment variables that are used in the build:
> > +#  - MAKEFLAGS: flags to pass to make (e.g., "-j8")
> > +#  - CCACHE_DIR: ccache directory (default: ~/.ccache)
> > +#
> > +# To run
> > +#
> > +# ~~~
> > +#   mkdir -p -m777 ~/my_results_dir
> > +#   docker run --privileged --rm -t -v ~/my_results_dir:/results:Z  \
> > +#  -v /tmp/.X11-unix:/tmp/.X11-unix freedesktop/mesa:piglit
> > +# ~~~
> > +#
> > +
> > +{{ $image := (or .IMAGE "freedesktop/mesa") }}
> > +{{ $parent_image := (or .PARENT "ubuntu:xenial") }}
> > +{{ $ccachedir := (or .Env.CCACHE_DIR "~/.ccache") }}
> > +
> > +FROM {{ $parent_image }}
> > +
> > +LABEL maintainer "Andres Gomez "
> > +
> > +USER root
> > +
> > +ENV LC_ALL=C.UTF-8
> > +
> > +RUN apt-get update  \
> > +  && apt-get -y --no-install-recommends

Re: [Piglit] [PATCH v4] travis: add docker based cmake build job

2018-02-05 Thread Dylan Baker 
I haven't looked too closely at this, but I like the idea of using docker to
allow running the cmake based job.

Acked-by: Dylan Baker 

Quoting Andres Gomez (2018-01-31 13:11:52)
> Until now we were only running the python unit tests.
> 
> It seems desirable to also check that the CMake based build compiles
> successfully. We do that now using docker.
> 
> The docker build can be tweaked with some environment variables and,
> also, be stored in the docker hub if desired. Check the changes for
> extra details regarding these variables.
> 
> v2: Removed other build possibilities other than just from inside
> Travis-CI, as suggested by Juan.
> v3: Replaced the "RELEASE" parameter to create the docker image with
> "PARENT" and removed some unneeded documentation after v2, as
> suggested by Juan.
> v4:
>   - Use DOCKER_PARENT, DOCKER_IMAGE and DOCKER_TAG variables from the
> custom Travis-CI setup to define the PARENT, IMAGE and TAG
> parameters during the docker image creation.
>   - Upload the image only if DOCKER_IMAGE and DOCKER_TAG are set.
> 
> Cc: Dylan Baker 
> Cc: Juan A. Suarez 
> Signed-off-by: Andres Gomez 
> Reviewed-by: Juan A. Suarez 
> ---
>  .travis.yml  | 47 --
>  docker/Rockerfile.piglit | 74 
> 
>  2 files changed, 118 insertions(+), 3 deletions(-)
>  create mode 100644 docker/Rockerfile.piglit
> 
> diff --git a/.travis.yml b/.travis.yml
> index b47829ff9..b4cbcca49 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -1,5 +1,16 @@
> +sudo: false
> +os: linux
>  language: python
> -cache: pip
> +cache:
> +  - ccache
> +  - pip
> +services:
> +  - docker
> +
> +env:
> +  global:
> +- BUILD=pytest
> +
>  matrix:
>include:
>  - python: 2.7
> @@ -12,7 +23,37 @@ matrix:
>env: TOX_ENV="py35-{generator,noaccel,accel-nix,streams}"
>  - python: 3.6
>env: TOX_ENV="py36-{generator,noaccel,accel-nix,streams}"
> +- env: BUILD=cmake
> +
>  install:
> -pip install tox
> +  - |
> +if [[ $BUILD == pytest ]]; then
> +  pip install tox
> +else
> +  wget 
> https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz
> +  tar xvf rocker-1.3.1-linux_amd64.tar.gz
> +  rm rocker-1.3.1-linux_amd64.tar.gz
> +fi
> +
> +before_script:
> +  - |
> +if [[ $BUILD != pytest ]]; then
> +  mkdir -p -m777 ~/.ccache
> +fi
> +
>  script:
> -- tox -e $TOX_ENV
> +  - |
> +if [[ $BUILD == pytest ]]; then
> +  tox -e $TOX_ENV
> +else
> +  ./rocker build -f docker/Rockerfile.piglit ${DOCKER_TAG:+--var 
> TAG=}${DOCKER_TAG} ${DOCKER_IMAGE:+--var IMAGE=}${DOCKER_IMAGE} 
> ${DOCKER_PARENT:+--var PARENT=}${DOCKER_PARENT} .
> +fi
> +
> +after_success:
> +  - |
> +if [[ $BUILD != pytest ]]; then
> +  if [[ -n $DOCKER_IMAGE && -n $DOCKER_TAG && -n $DOCKER_USERNAME && 
> $TRAVIS_BRANCH == master ]]; then
> +docker login -u "${DOCKER_USERNAME}" -p "${DOCKER_PASSWORD}"
> +docker push "${DOCKER_IMAGE}":"${DOCKER_TAG}"
> +  fi
> +fi
> diff --git a/docker/Rockerfile.piglit b/docker/Rockerfile.piglit
> new file mode 100644
> index 0..0d8f8bb45
> --- /dev/null
> +++ b/docker/Rockerfile.piglit
> @@ -0,0 +1,74 @@
> +#
> +# This builds Piglit.
> +#
> +# ~~~
> +#  rocker build -f Rockerfile.piglit [--attach]  
>\
> +#[--var IMAGE=freedesktop/mesa] # freedesktop/mesa, myrepo/myproyect, 
> ...   \
> +#[--var TAG=piglit] # piglit-17.0, piglit-13.0, ...  
>\
> +#[--var PARENT=ubuntu:xenial]   # ubuntu:xenial, freedesktop/mesa:17.3, 
> ...
> +# ~~~
> +#
> +# Environment variables that are used in the build:
> +#  - MAKEFLAGS: flags to pass to make (e.g., "-j8")
> +#  - CCACHE_DIR: ccache directory (default: ~/.ccache)
> +#
> +# To run
> +#
> +# ~~~
> +#   mkdir -p -m777 ~/my_results_dir
> +#   docker run --privileged --rm -t -v ~/my_results_dir:/results:Z  \
> +#  -v /tmp/.X11-unix:/tmp/.X11-unix freedesktop/mesa:piglit
> +# ~~~
> +#
> +
> +{{ $image := (or .IMAGE "freedesktop/mesa") }}
> +{{ $parent_image := (or .PARENT "ubuntu:xenial") }}
> +{{ $ccachedir := (or .Env.CCACHE_DIR "~/.ccache") }}
> +
> +FROM {{ $parent_image }}
> +
> +LABEL maintainer "Andres Gomez "
> +
> +USER root
> +
> +ENV LC_ALL=C.UTF-8
> +
> +RUN apt-get update  \
> +  && apt-get -y --no-install-recommends install sudo gcc g++ ccache \
> +git pkg-config bash-completion cmake\
> +libz-dev libpng-dev libgl-dev libegl1-mesa-dev libwaffle-dev\
> +python3-setuptools python3-pip  \
> +  && rm -fr /var/lib/apt/lists/*
> +
> +RUN pip3 install numpy six mako
> +
> +RUN getent passwd local > /dev/null || adduser --gecos "" local && passwd -d 
> local && adduser local sudo
> +
> +USER local
> +
>

Re: [Piglit] [PATCH v4] travis: add docker based cmake build job

2018-02-03 Thread Eric Anholt 
Andres Gomez  writes:

> Until now we were only running the python unit tests.
>
> It seems desirable to also check that the CMake based build compiles
> successfully. We do that now using docker.
>
> The docker build can be tweaked with some environment variables and,
> also, be stored in the docker hub if desired. Check the changes for
> extra details regarding these variables.
>
> v2: Removed other build possibilities other than just from inside
> Travis-CI, as suggested by Juan.
> v3: Replaced the "RELEASE" parameter to create the docker image with
> "PARENT" and removed some unneeded documentation after v2, as
> suggested by Juan.
> v4:
>   - Use DOCKER_PARENT, DOCKER_IMAGE and DOCKER_TAG variables from the
> custom Travis-CI setup to define the PARENT, IMAGE and TAG
> parameters during the docker image creation.
>   - Upload the image only if DOCKER_IMAGE and DOCKER_TAG are set.
>
> Cc: Dylan Baker 
> Cc: Juan A. Suarez 
> Signed-off-by: Andres Gomez 
> Reviewed-by: Juan A. Suarez 

If this build has proven to be fast enough, I should probably take a
look at doing this for the xserver as well.  (For xserver, the issue was
all the package installs per Travis run, so I have an image I manage on
dockerhub instead)

Acked-by: Eric Anholt 


signature.asc
Description: PGP signature
___
Piglit mailing list
Piglit@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/piglit

[Piglit] [PATCH v4] travis: add docker based cmake build job

2018-01-31 Thread Andres Gomez 
Until now we were only running the python unit tests.

It seems desirable to also check that the CMake based build compiles
successfully. We do that now using docker.

The docker build can be tweaked with some environment variables and,
also, be stored in the docker hub if desired. Check the changes for
extra details regarding these variables.

v2: Removed other build possibilities other than just from inside
Travis-CI, as suggested by Juan.
v3: Replaced the "RELEASE" parameter to create the docker image with
"PARENT" and removed some unneeded documentation after v2, as
suggested by Juan.
v4:
  - Use DOCKER_PARENT, DOCKER_IMAGE and DOCKER_TAG variables from the
custom Travis-CI setup to define the PARENT, IMAGE and TAG
parameters during the docker image creation.
  - Upload the image only if DOCKER_IMAGE and DOCKER_TAG are set.

Cc: Dylan Baker 
Cc: Juan A. Suarez 
Signed-off-by: Andres Gomez 
Reviewed-by: Juan A. Suarez 
---
 .travis.yml  | 47 --
 docker/Rockerfile.piglit | 74 
 2 files changed, 118 insertions(+), 3 deletions(-)
 create mode 100644 docker/Rockerfile.piglit

diff --git a/.travis.yml b/.travis.yml
index b47829ff9..b4cbcca49 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,16 @@
+sudo: false
+os: linux
 language: python
-cache: pip
+cache:
+  - ccache
+  - pip
+services:
+  - docker
+
+env:
+  global:
+- BUILD=pytest
+
 matrix:
   include:
 - python: 2.7
@@ -12,7 +23,37 @@ matrix:
   env: TOX_ENV="py35-{generator,noaccel,accel-nix,streams}"
 - python: 3.6
   env: TOX_ENV="py36-{generator,noaccel,accel-nix,streams}"
+- env: BUILD=cmake
+
 install:
-pip install tox
+  - |
+if [[ $BUILD == pytest ]]; then
+  pip install tox
+else
+  wget 
https://github.com/grammarly/rocker/releases/download/1.3.1/rocker-1.3.1-linux_amd64.tar.gz
+  tar xvf rocker-1.3.1-linux_amd64.tar.gz
+  rm rocker-1.3.1-linux_amd64.tar.gz
+fi
+
+before_script:
+  - |
+if [[ $BUILD != pytest ]]; then
+  mkdir -p -m777 ~/.ccache
+fi
+
 script:
-- tox -e $TOX_ENV
+  - |
+if [[ $BUILD == pytest ]]; then
+  tox -e $TOX_ENV
+else
+  ./rocker build -f docker/Rockerfile.piglit ${DOCKER_TAG:+--var 
TAG=}${DOCKER_TAG} ${DOCKER_IMAGE:+--var IMAGE=}${DOCKER_IMAGE} 
${DOCKER_PARENT:+--var PARENT=}${DOCKER_PARENT} .
+fi
+
+after_success:
+  - |
+if [[ $BUILD != pytest ]]; then
+  if [[ -n $DOCKER_IMAGE && -n $DOCKER_TAG && -n $DOCKER_USERNAME && 
$TRAVIS_BRANCH == master ]]; then
+docker login -u "${DOCKER_USERNAME}" -p "${DOCKER_PASSWORD}"
+docker push "${DOCKER_IMAGE}":"${DOCKER_TAG}"
+  fi
+fi
diff --git a/docker/Rockerfile.piglit b/docker/Rockerfile.piglit
new file mode 100644
index 0..0d8f8bb45
--- /dev/null
+++ b/docker/Rockerfile.piglit
@@ -0,0 +1,74 @@
+#
+# This builds Piglit.
+#
+# ~~~
+#  rocker build -f Rockerfile.piglit [--attach]
 \
+#[--var IMAGE=freedesktop/mesa] # freedesktop/mesa, myrepo/myproyect, ...  
 \
+#[--var TAG=piglit] # piglit-17.0, piglit-13.0, ...
 \
+#[--var PARENT=ubuntu:xenial]   # ubuntu:xenial, freedesktop/mesa:17.3, ...
+# ~~~
+#
+# Environment variables that are used in the build:
+#  - MAKEFLAGS: flags to pass to make (e.g., "-j8")
+#  - CCACHE_DIR: ccache directory (default: ~/.ccache)
+#
+# To run
+#
+# ~~~
+#   mkdir -p -m777 ~/my_results_dir
+#   docker run --privileged --rm -t -v ~/my_results_dir:/results:Z  \
+#  -v /tmp/.X11-unix:/tmp/.X11-unix freedesktop/mesa:piglit
+# ~~~
+#
+
+{{ $image := (or .IMAGE "freedesktop/mesa") }}
+{{ $parent_image := (or .PARENT "ubuntu:xenial") }}
+{{ $ccachedir := (or .Env.CCACHE_DIR "~/.ccache") }}
+
+FROM {{ $parent_image }}
+
+LABEL maintainer "Andres Gomez "
+
+USER root
+
+ENV LC_ALL=C.UTF-8
+
+RUN apt-get update  \
+  && apt-get -y --no-install-recommends install sudo gcc g++ ccache \
+git pkg-config bash-completion cmake\
+libz-dev libpng-dev libgl-dev libegl1-mesa-dev libwaffle-dev\
+python3-setuptools python3-pip  \
+  && rm -fr /var/lib/apt/lists/*
+
+RUN pip3 install numpy six mako
+
+RUN getent passwd local > /dev/null || adduser --gecos "" local && passwd -d 
local && adduser local sudo
+
+USER local
+
+{{ if .Env.MAKEFLAGS }}
+ENV MAKEFLAGS={{ .Env.MAKEFLAGS }}
+{{ end }}
+
+WORKDIR /home/local
+
+MOUNT {{ $ccachedir }}:/home/local/.ccache:Z
+
+RUN sudo chown -R local:local /home/local/.ccache
+
+ENV PATH=/usr/lib/ccache:$PATH
+
+ADD . /home/local/piglit
+RUN sudo chown -R local:local /home/local/piglit
+
+WORKDIR /home/local/piglit
+
+ATTACH [ "/bin/bash" ]
+
+RUN cmake . && cmake --build .
+
+VOLUME /results
+
+{{ if .TAG }}
+TAG {{ $image }}:{{ .TAG }}
+{{ end }}
-- 
2.15.1

__