Elukey has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/328381 )
Change subject: Add another test to run Varnishkafka with Valgrind
......................................................................
Add another test to run Varnishkafka with Valgrind
Bug: T147438
Change-Id: I745588655af75e636b921ba4fd62244e24a2cbfb
---
M Dockerfile
M README.md
A run_valgrind_test
3 files changed, 92 insertions(+), 2 deletions(-)
git pull
ssh://gerrit.wikimedia.org:29418/operations/software/varnish/varnishkafka/testing
refs/changes/81/328381/1
diff --git a/Dockerfile b/Dockerfile
index 8a086c0..26c78ae 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -48,12 +48,14 @@
# Update the repository sources list
RUN apt-get update && apt-get -y dist-upgrade
-
# Install Varnish 4 and Apache 2.4
RUN apt-get install -y varnish apache2 php5-fpm
# Install Varnishkafka dev libs
RUN apt-get install -y librdkafka-dev libvarnishapi-dev zlib1g-dev libyajl-dev
git gcc make python
+
+# Install Valgrind
+RUN apt-get install -y valgrind
# Copy configuration files
RUN mkdir -p /var/www/test
@@ -71,9 +73,11 @@
COPY ${VARNISHKAFKA_SOURCE_PATH}/* /tmp/varnishkafka/
COPY ${SERVICES_CONFIG_PATH}/varnishkafka.conf
/tmp/varnishkafka/varnishkafka.conf
-# Copy the test script
+# Copy the test scripts
COPY run_test /run_test
+COPY run_valgrind_test /run_valgrind_test
RUN chmod +x /run_test
+RUN chmod +x /run_valgrind_test
# Copy validation script
COPY check_output.py /tmp/varnishkafka/check_vk_output.py
diff --git a/README.md b/README.md
index 3071021..a5a9383 100644
--- a/README.md
+++ b/README.md
@@ -36,6 +36,10 @@
* Logs periodically system metrics related to CPU/Memory utilization and load.
* Checks the Varnishkafka output and report any inconsistency.
+## run_valgrind_test
+This bash script is the same as the above one with the important exception
that Varnishkafka
+will run through Valgrind. The final report is the Valgrind output for memory
leak checks.
+
# Instructions
## Building
diff --git a/run_valgrind_test b/run_valgrind_test
new file mode 100644
index 0000000..d5a697d
--- /dev/null
+++ b/run_valgrind_test
@@ -0,0 +1,82 @@
+#!/bin/bash
+
+# Copyright © 2016 Luca Toscano
+# Copyright © 2016 Wikimedia Foundation, Inc.
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+##################################################################
+# Script that performs the following things:
+# 1) Configure Apache modules loaded.
+# 2) Spins up Apache, Varnish and PHP-FPM.
+# 3) Build and Install Varnishkafka.
+# 4) Runs Varnishkafka via Valgrind and nohup.
+# 5) Return the Valgrind output after certain amount
+# of HTTP requests. Since it is not meant to be a load test
+# it is sufficient only a simple bash loop with curl.
+#
+# GOAL: Memory leak checks for Varnishkafka.
+#
+# Assumptions:
+# 1) The OS is Debian Jessie (especially for the Apache config).
+# 2) Varnishkafka logs to stdout in JSON format (Kafka not tested).
+# 3) Apache, Varnish, PHP-FPM are already configured.
+# 4) Varnishkafka sources are located in /tmp/varnishkafka.
+# 5) Curl and Python are available.
+###################################################################
+
+set -e
+
+#Number of runs to perform
+RUNS=100
+
+if [ $1 -gt 0 ]
+then
+ RUNS=$1
+fi
+
+# Compile and install Varnishkafka
+cd /tmp/varnishkafka
+make && make install
+
+# Enable Apache httpd modules
+a2enmod proxy
+a2enmod proxy_fcgi
+
+# Start the processing chain Varnish -> Apache -> php-fpm
+/usr/sbin/service varnish start
+/usr/sbin/service php5-fpm start
+/usr/sbin/service apache2 start
+
+VALGRIND_ARGS="--log-file=/tmp/varnishkafka/valgrind.log --leak-check=full
--show-leak-kinds=all"
+# Start Varnishkafka in the background with Valgrind, logging to a known file.
+# Note: stdout tuned to be line buffered
+nohup stdbuf -oL /usr/bin/valgrind $VALGRIND_ARGS /usr/local/bin/varnishkafka
-S /tmp/varnishkafka/varnishkafka.conf > /dev/null &
+
+echo "Running curl to generate some fake traffic (all MISSes thanks to the
variable query string)"
+for run in $(seq 1 $RUNS);
+ do
+ echo "Run: $run";
+ curl -i -s http://localhost/test.php?$run --header "Host: test.com"
--header "Referer: hello.com?advice=readwikipedia" | wc -l | xargs echo -e
"\tHTML Response lines:";
+done
+
+# Kill Varnishkafka to force Valgrind to emit its output
+echo -e "\n\nKilling Varnishkafka"
+kill $(pgrep -f varnishkafka)
+
+echo -e "\n\nWait a couple of seconds to ensure that the Valgrind output is
flushed"
+sleep 3
+
+echo -e "\n\nValgrind output:"
+cat /tmp/varnishkafka/valgrind.log
+
--
To view, visit https://gerrit.wikimedia.org/r/328381
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I745588655af75e636b921ba4fd62244e24a2cbfb
Gerrit-PatchSet: 1
Gerrit-Project: operations/software/varnish/varnishkafka/testing
Gerrit-Branch: master
Gerrit-Owner: Elukey <ltosc...@wikimedia.org>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
