[
https://issues.apache.org/jira/browse/AVRO-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639978#comment-16639978
]
ASF GitHub Bot commented on AVRO-2229:
--------------------------------------
cutting closed pull request #336: [AVRO-2229] Test using a Docker image
URL: https://github.com/apache/avro/pull/336
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/BUILD.txt b/BUILD.md
similarity index 67%
rename from BUILD.txt
rename to BUILD.md
index 223bc4d04..882338aee 100644
--- a/BUILD.txt
+++ b/BUILD.md
@@ -1,6 +1,6 @@
-Apache Avro Build Instructions
+# Apache Avro Build Instructions
-REQUIREMENTS
+## Requirements
The following packages must be installed before Avro can be built:
@@ -18,13 +18,18 @@ The following packages must be installed before Avro can be
built:
IO::String, Object::Tiny, Compress::ZLib, Test::More,
Test::Exception, Test::Pod
- Apache Ant 1.7
- - Apache Forrest 0.8 (for documentation)
+ - Apache Forrest 0.9 (for documentation)
- md5sum, sha1sum, used by top-level dist target
To simplify this, you can run a Docker container with all the above
-dependencies installed by installing docker.io and typing:
+dependencies installed by installing Docker and run:
- ./build.sh docker
+```bash
+./build.sh docker
+docker@539f6535c9db:~/avro$ cd lang/java/
+docker@539f6535c9db:~/avro/lang/java$ ./build.sh test
+[INFO] Scanning for projects...
+```
When this completes you will be in a shell running in the
container. Building the image the first time may take a while (20
@@ -36,11 +41,22 @@ The working directory in the container is mounted from your
host. This
allows you to access the files in your Avro development tree from the
Docker container.
-BUILDING
+## Building
Once the requirements are installed (or from the Docker container),
build.sh can be used as follows:
- './build.sh test' runs tests for all languages
- './build.sh dist' creates all release distribution files in dist/
- './build.sh clean' removes all generated artifacts
+```
+./build.sh test # runs tests for all languages
+./build.sh dist # creates all release distribution files in dist/
+./build.sh clean # removes all generated artifacts
+```
+
+## Testing
+
+Testing is done with the same Docker container as mentioned in the building
+step. The difference is that it will do clean run of the full test suite:
+
+```bash
+./build.sh docker-test
+```
diff --git a/build.sh b/build.sh
index 255e46af7..8810d6bfa 100755
--- a/build.sh
+++ b/build.sh
@@ -22,7 +22,7 @@ cd `dirname "$0"` # connect to root
VERSION=`cat share/VERSION.txt`
function usage {
- echo "Usage: $0 {test|dist|sign|clean|docker|rat|githooks}"
+ echo "Usage: $0 {test|dist|sign|clean|docker|rat|githooks|docker-test}"
exit 1
}
@@ -197,7 +197,7 @@ do
;;
docker)
- docker build -t avro-build share/docker
+ docker build -t avro-build -f share/docker/Dockerfile .
if [ "$(uname -s)" == "Linux" ]; then
USER_NAME=${SUDO_USER:=$USER}
USER_ID=$(id -u $USER_NAME)
@@ -217,13 +217,13 @@ UserSpecificDocker
# within the container and use the result on your normal
# system. And this also is a significant speedup in subsequent
# builds because the dependencies are downloaded only once.
- docker run --rm=true -t -i \
+ docker run --rm -t -i \
-v ${PWD}:/home/${USER_NAME}/avro \
-w /home/${USER_NAME}/avro \
-v ${HOME}/.m2:/home/${USER_NAME}/.m2 \
-v ${HOME}/.gnupg:/home/${USER_NAME}/.gnupg \
-u ${USER_NAME} \
- avro-build-${USER_NAME}
+ avro-build-${USER_NAME} bash
;;
rat)
@@ -236,6 +236,11 @@ UserSpecificDocker
find .git/hooks/ -type f | fgrep -v sample | xargs chmod 755
;;
+ docker-test)
+ docker build -t avro-test -f share/docker/Dockerfile .
+ docker run --rm -v ${PWD}:/avro/ avro-test
+ ;;
+
*)
usage
;;
diff --git a/lang/csharp/src/apache/codegen/app.config
b/lang/csharp/src/apache/codegen/app.config
index ae2479060..e8a42543c 100755
--- a/lang/csharp/src/apache/codegen/app.config
+++ b/lang/csharp/src/apache/codegen/app.config
@@ -15,5 +15,8 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
-<configuration>
-<startup><supportedRuntime version="v2.0.50727"/></startup></configuration>
+<configuration useLegacyV2RuntimeActivationPolicy="true">
+ <startup>
+ <supportedRuntime version="v4.0.30319" />
+ </startup>
+</configuration>
diff --git a/lang/csharp/src/apache/perf/app.config
b/lang/csharp/src/apache/perf/app.config
index ae2479060..e8a42543c 100755
--- a/lang/csharp/src/apache/perf/app.config
+++ b/lang/csharp/src/apache/perf/app.config
@@ -15,5 +15,8 @@
See the License for the specific language governing permissions and
limitations under the License.
-->
-<configuration>
-<startup><supportedRuntime version="v2.0.50727"/></startup></configuration>
+<configuration useLegacyV2RuntimeActivationPolicy="true">
+ <startup>
+ <supportedRuntime version="v4.0.30319" />
+ </startup>
+</configuration>
diff --git a/lang/java/build.sh b/lang/java/build.sh
index bf1a4fb9d..76f7f579d 100755
--- a/lang/java/build.sh
+++ b/lang/java/build.sh
@@ -43,7 +43,7 @@ function do_dist() {
case "$target" in
test)
- mvn test
+ mvn -B test
;;
dist)
diff --git a/lang/py3/setup.py b/lang/py3/setup.py
old mode 100644
new mode 100755
diff --git a/share/docker/Dockerfile b/share/docker/Dockerfile
index 535ae318f..de09220a1 100644
--- a/share/docker/Dockerfile
+++ b/share/docker/Dockerfile
@@ -21,52 +21,88 @@ FROM java:8-jdk
WORKDIR /root
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
# Add the repository for node.js 4.x
RUN curl -sL https://deb.nodesource.com/setup_4.x | bash -
# Install dependencies from packages
-RUN apt-get update && apt-get install --no-install-recommends -y \
- git subversion curl ant make maven \
- gcc cmake libjansson-dev asciidoc source-highlight \
- g++ flex bison libboost-all-dev doxygen \
- mono-devel mono-gmcs nunit \
- nodejs \
- perl \
- php5 phpunit php5-gmp bzip2 \
- python python-setuptools python3-setuptools \
- ruby ruby-dev rake \
- libsnappy1 libsnappy-dev
+RUN apt-get update && \
+ apt-get install --no-install-recommends -y \
+ ant \
+ asciidoc \
+ bison \
+ bzip2 \
+ cmake \
+ curl \
+ doxygen \
+ flex \
+ g++ \
+ gcc \
+ git \
+ libboost-all-dev \
+ libfontconfig1-dev \
+ libfreetype6-dev \
+ libglib2.0-dev \
+ libjansson-dev \
+ libsnappy-dev \
+ libsnappy1 \
+ make \
+ maven \
+ mono-devel \
+ nodejs \
+ nunit \
+ perl \
+ php5 \
+ php5-gmp \
+ phpunit \
+ python \
+ python-setuptools \
+ python3-setuptools \
+ rake \
+ ruby \
+ ruby-dev \
+ source-highlight \
+ subversion && \
+ apt-get clean && \
+ rm -rf /var/lib/apt/lists/*
# Install Forrest in /usr/local/apache-forrest
-# Download
-RUN cd /usr/local/ && wget
"http://www.apache.org/dyn/closer.lua?action=download&filename=/forrest/apache-forrest-0.9-sources.tar.gz";
-O "apache-forrest-0.9-sources.tar.gz"
-RUN cd /usr/local/ && wget
"http://www.apache.org/dyn/closer.lua?action=download&filename=/forrest/apache-forrest-0.9-dependencies.tar.gz";
-O "apache-forrest-0.9-dependencies.tar.gz"
-
-# Unpack Apache Forrest
-RUN cd /usr/local/ && \
- tar xzf apache-forrest-0.9-sources.tar.gz && \
- tar xzf apache-forrest-0.9-dependencies.tar.gz && \
- mv apache-forrest-0.9 apache-forrest
-RUN cd /usr/local/apache-forrest/main && ./build.sh
+RUN curl -L -s
"http://www.apache.org/dyn/closer.lua?action=download&filename=/forrest/apache-forrest-0.9-sources.tar.gz";
| tar -xzf - -C /usr/local/ && \
+ curl -L -s
"http://www.apache.org/dyn/closer.lua?action=download&filename=/forrest/apache-forrest-0.9-dependencies.tar.gz";
| tar -xzf - -C /usr/local/ && \
+ mv /usr/local/apache-forrest-0.9 /usr/local/apache-forrest && \
+ cd /usr/local/apache-forrest/main && \
+ ./build.sh
# The solution for https://issues.apache.org/jira/browse/PIG-3906
-RUN mkdir -p /usr/local/apache-forrest/plugins && chmod a+rwX -R
/usr/local/apache-forrest/plugins
-RUN mkdir -p /usr/local/apache-forrest/build/plugins && chmod a+rwX -R
/usr/local/apache-forrest/build/plugins
-
# Configure where forrest can be found
-RUN echo 'forrest.home=/usr/local/apache-forrest' > build.properties
+RUN mkdir -p /usr/local/apache-forrest/plugins && chmod a+rwX -R
/usr/local/apache-forrest/plugins && \
+ mkdir -p /usr/local/apache-forrest/build/plugins && chmod a+rwX -R
/usr/local/apache-forrest/build/plugins && \
+ echo 'forrest.home=/usr/local/apache-forrest' > build.properties
+
ENV FORREST_HOME /usr/local/apache-forrest
# Install Perl modules
-RUN curl -L http://cpanmin.us | perl - --self-upgrade # non-interactive cpan
-RUN cpanm install Module::Install Module::Install::ReadmeFromPod \
+RUN curl -L http://cpanmin.us | perl - --self-upgrade && \
+ cpanm install Module::Install Module::Install::ReadmeFromPod \
Module::Install::Repository \
Math::BigInt JSON::XS Try::Tiny Regexp::Common Encode \
IO::String Object::Tiny Compress::Zlib Test::More \
Test::Exception Test::Pod
+# Install mono modules
+RUN mkdir -p /tmp/nunit/ && \
+ cd /tmp/nunit/ && \
+ curl -L -s -o nunit.zip
https://github.com/nunit-legacy/nunitv2/releases/download/2.7.0/NUnit-2.7.0.zip
&& \
+ unzip nunit.zip && \
+ rm nunit.zip
+
+ENV MONO_PATH /tmp/nunit/bin/lib
+
# Install Ruby modules
RUN gem install echoe yajl-ruby multi_json snappy
# Install global Node modules
RUN npm install -g grunt-cli
+
+CMD ["/avro/share/docker/run-tests.sh"]
diff --git a/share/docker/run-tests.sh b/share/docker/run-tests.sh
new file mode 100755
index 000000000..e9919a7f6
--- /dev/null
+++ b/share/docker/run-tests.sh
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+headline(){
+ echo -e
"\e[1;34m#################################################################"
+ echo -e "##### $1 \e[1;37m"
+ echo -e
"\e[1;34m#################################################################\e[0m"
+}
+
+set -e
+
+for lang in /avro/lang/*/
+do
+ headline "Run tests: $lang"
+ cd "$lang"
+
+ if [[ "$lang" = *"c++"* ]]; then
+ # The current cpp tests are failing:
+ # https://issues.apache.org/jira/projects/AVRO/issues/AVRO-2230
+ ./build.sh test || true
+ else
+ ./build.sh test
+ fi
+done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Ability to test using a Docker image
> ------------------------------------
>
> Key: AVRO-2229
> URL: https://issues.apache.org/jira/browse/AVRO-2229
> Project: Avro
> Issue Type: Improvement
> Affects Versions: 1.8.2
> Reporter: Fokko Driesprong
> Priority: Major
> Fix For: 1.9.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
