I've added some tests to check polipo's behaviour with Cache-Control values.
I've attached a patch which includes the bug fixing and these tests.
This patch have been tested on automated-testing2 branch of
https://github.com/wmanley/polipo. Thanks
This transmission contains information that may be confidential and contain
personal views which are not necessarily those of YouView TV Ltd. YouView TV
Ltd (Co No:7308805) is a limited liability company registered in England and
Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower Thames
Street, London, EC3R 6YT. For details see our web site at http://www.youview.com
From 6ea766a3046c0fcff64d5ae0cf69a54da531aa46 Mon Sep 17 00:00:00 2001
From: Ruben Alleres <[email protected]>
Date: Fri, 1 Mar 2013 12:14:01 +0000
Subject: [PATCH] Fix Polipo ignores cache control values when it has restarted.
The max-age value of Cache-control could be received in a http header.
Polipo web cache proxy should be set this value as the maximum time
that this page is stored in cache. When max-age has elapsed the cache is
stale and it should be refreshed making a new request to the server.
Polipo's is saving the cache control values in memory but they are not
saved on disk doing that polipo doesn't know the cache control values when
is restarted and setting them with a default value.
httpPrintCacheControl is the polipo's function which writes the values from a
CacheControlRec object in a buffer. It has two callers:
- httpWriteRequest which does pass in a CacheControlRec object.
- httpWriteObjectHeaders which did pass a NULL object rather a
CacheControlRec object.
httpWriteObjectHeaders is called when polipo writes the header values on disk
cache so
some cache control values was never written on disk. It has been updated to
write
cache-control values on disk.
- Updated the call to httpPrintCacheControl from httpWriteObjectHeaders passing
an updated
CacheControlRec object as parameter.
- Updated validateEntry function to fill the memory cache-control values when
they has
been read from disk cache.
- Added test cases in polipo to check its behaviour when it receives a header
which contains
Cache-Control values:
- Simple test cases using max-age, min-fresh and s-maxage values.
- Test case for multiple values in the same header: i.e: max-age=3,
min-fresh=20
(it expires when the minimum time has elapsed)
---
diskcache.c | 2 +
http.c | 9 +++++-
test/expect_expire/cache_control_maxage.header | 4 +++
test/expect_expire/cache_control_maxage.payload | 1 +
test/expect_expire/cache_control_minfresh.header | 4 +++
test/expect_expire/cache_control_minfresh.payload | 1 +
test/expect_expire/cache_control_multiple.header | 4 +++
test/expect_expire/cache_control_multiple.payload | 1 +
test/expect_expire/cache_control_smaxage.header | 4 +++
test/expect_expire/cache_control_smaxage.payload | 1 +
test/run-test.sh | 30 ++++++++++++++++++++-
11 files changed, 59 insertions(+), 2 deletions(-)
create mode 100644 test/expect_expire/cache_control_maxage.header
create mode 100644 test/expect_expire/cache_control_maxage.payload
create mode 100644 test/expect_expire/cache_control_minfresh.header
create mode 100644 test/expect_expire/cache_control_minfresh.payload
create mode 100644 test/expect_expire/cache_control_multiple.header
create mode 100644 test/expect_expire/cache_control_multiple.payload
create mode 100644 test/expect_expire/cache_control_smaxage.header
create mode 100644 test/expect_expire/cache_control_smaxage.payload
diff --git a/diskcache.c b/diskcache.c
index 4dbaa99..d11f15b 100644
--- a/diskcache.c
+++ b/diskcache.c
@@ -1035,6 +1035,8 @@ validateEntry(ObjectPtr object, int fd,
dirty = 1;
object->cache_control |= cache_control.flags;
+ object->max_age = cache_control.max_age;
+ object->s_maxage = cache_control.s_maxage;
if(object->age < 0) object->age = object->date;
if(object->age < 0) object->age = 0; /* a long time ago */
diff --git a/http.c b/http.c
index db35bbf..4d96a1b 100644
--- a/http.c
+++ b/http.c
@@ -286,6 +286,13 @@ httpWriteObjectHeaders(char *buf, int offset, int len,
ObjectPtr object, int from, int to)
{
int n = offset;
+ CacheControlRec cache_control;
+
+ cache_control.flags = object->flags;
+ cache_control.max_age = object->max_age;
+ cache_control.s_maxage = object->s_maxage;
+ cache_control.max_stale = -1;
+ cache_control.min_fresh = -1;
if(from <= 0 && to < 0) {
if(object->length >= 0) {
@@ -351,7 +358,7 @@ httpWriteObjectHeaders(char *buf, int offset, int len,
}
n = httpPrintCacheControl(buf, n, len,
- object->cache_control, NULL);
+ object->cache_control, &cache_control);
if(n < 0)
goto fail;
diff --git a/test/expect_expire/cache_control_maxage.header
b/test/expect_expire/cache_control_maxage.header
new file mode 100644
index 0000000..5098549
--- /dev/null
+++ b/test/expect_expire/cache_control_maxage.header
@@ -0,0 +1,4 @@
+HTTP/1.1 200 OK
+Content-type: text/html
+Cache-Control: max-age=3
+
diff --git a/test/expect_expire/cache_control_maxage.payload
b/test/expect_expire/cache_control_maxage.payload
new file mode 100644
index 0000000..ca1ab04
--- /dev/null
+++ b/test/expect_expire/cache_control_maxage.payload
@@ -0,0 +1 @@
+<html><head><body>Testing Cache-Control values</body></head></html>
diff --git a/test/expect_expire/cache_control_minfresh.header
b/test/expect_expire/cache_control_minfresh.header
new file mode 100644
index 0000000..6dbbbcf
--- /dev/null
+++ b/test/expect_expire/cache_control_minfresh.header
@@ -0,0 +1,4 @@
+HTTP/1.1 200 OK
+Content-type: text/html
+Cache-Control: min-fresh=3
+
diff --git a/test/expect_expire/cache_control_minfresh.payload
b/test/expect_expire/cache_control_minfresh.payload
new file mode 100644
index 0000000..ca1ab04
--- /dev/null
+++ b/test/expect_expire/cache_control_minfresh.payload
@@ -0,0 +1 @@
+<html><head><body>Testing Cache-Control values</body></head></html>
diff --git a/test/expect_expire/cache_control_multiple.header
b/test/expect_expire/cache_control_multiple.header
new file mode 100644
index 0000000..99715da
--- /dev/null
+++ b/test/expect_expire/cache_control_multiple.header
@@ -0,0 +1,4 @@
+HTTP/1.1 200 OK
+Content-type: text/html
+Cache-Control: max-age=3 min-fresh=20
+
diff --git a/test/expect_expire/cache_control_multiple.payload
b/test/expect_expire/cache_control_multiple.payload
new file mode 100644
index 0000000..ca1ab04
--- /dev/null
+++ b/test/expect_expire/cache_control_multiple.payload
@@ -0,0 +1 @@
+<html><head><body>Testing Cache-Control values</body></head></html>
diff --git a/test/expect_expire/cache_control_smaxage.header
b/test/expect_expire/cache_control_smaxage.header
new file mode 100644
index 0000000..d0ce359
--- /dev/null
+++ b/test/expect_expire/cache_control_smaxage.header
@@ -0,0 +1,4 @@
+HTTP/1.1 200 OK
+Content-type: text/html
+Cache-Control: s-maxage=3
+
diff --git a/test/expect_expire/cache_control_smaxage.payload
b/test/expect_expire/cache_control_smaxage.payload
new file mode 100644
index 0000000..ca1ab04
--- /dev/null
+++ b/test/expect_expire/cache_control_smaxage.payload
@@ -0,0 +1 @@
+<html><head><body>Testing Cache-Control values</body></head></html>
diff --git a/test/run-test.sh b/test/run-test.sh
index 5042291..2211673 100755
--- a/test/run-test.sh
+++ b/test/run-test.sh
@@ -36,6 +36,7 @@ stop_polipo () {
run_test_case () {
FILENAME_ROOT=$1
EXPECTED_REQUESTS_SERVED=$2
+ [ "$#" -gt 2 ] && TIME_TO_EXPIRE=`expr $3 + 1`
PAYLOAD=${FILENAME_ROOT}.payload
HEADER=${FILENAME_ROOT}.header
@@ -57,16 +58,18 @@ run_test_case () {
for n in $(seq 1 10)
do
curl -o "${SAVE_LOCATION}"
"http://localhost:${HTTP_SERVER_PORT}/"; 2>/dev/null
+ [[ ! -z "${TIME_TO_EXPIRE:-}" && "$n" -eq 5 ]] && sleep
$TIME_TO_EXPIRE
cmp "${PAYLOAD}" "${SAVE_LOCATION}" || fail "Files don't match"
rm "${SAVE_LOCATION}"
done
stop_polipo
-
+ [[ ! -z "${TIME_TO_EXPIRE:-}" ]] && sleep $TIME_TO_EXPIRE
for n in $(seq 1 10)
do
(
start_polipo
curl -o "${SAVE_LOCATION}"
"http://localhost:${HTTP_SERVER_PORT}/"; 2>/dev/null
+ [[ ! -z "${TIME_TO_EXPIRE:-}" && "$n" -eq 5 ]] && sleep
$TIME_TO_EXPIRE
stop_polipo
)
cmp "${PAYLOAD}" "${SAVE_LOCATION}" || fail "Files don't match"
@@ -91,6 +94,31 @@ do
run_test_case ${HEADER%%.header} 20
done
+# Check polipo behaviour caching data with Cache-Control values
+# in header related with expiration time.
+# An expect_expire directory was provide with
+# different header test cases.
+# run_test_case make 20 request:
+# - The 10 first request are consecutives
+# (Polipo won't be closed between requests).
+# - In the next 10 request polipo will be started before
+# every request and closed after. This allow check the
+# polipo behaviour when is restarted and there are cached.
+# data with expire time specified.
+# The third argument of run_test_case is the duration in seconds
+# when the cached data time will expire and it must be its value must
+# be the same than the Cache-control value field (max-age, min-fresh or
+# s-maxage). The test will sleep after 5th and 15th request and between
+# the 10 first test cases and the 10 next ones during the time to cause
+# cached data time expiration.
+# 4 request will be made to the server:
+# - Initial request. (response is cached)
+# - In the 6th,11st and 16th requests (after cached data expires)
+for HEADER in ${POLIPO_TEST_DIR}/expect_expire/*.header
+do
+ run_test_case ${HEADER%%.header} 4 3
+done
+
rm -R "${TMP_DIR}"
echo SUCCESS
--
1.7.1
------------------------------------------------------------------------------
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_feb
_______________________________________________
Polipo-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/polipo-users
