[Bug 1336742] Re: Caching responses with Vary header
This bug was fixed in the package squid3 - 3.3.8-1ubuntu6.3 --- squid3 (3.3.8-1ubuntu6.3) trusty-proposed; urgency=low * d/patches/fix-caching-vary-header.patch: Added upstream patch for the bug which prevented squid from caching responses with Vary header. (LP: #1336742) Based on work by Oleg Strikov. -- Rolf Leggewie f...@rolf.leggewie.biz Wed, 01 Jul 2015 15:25:59 -0700 ** Changed in: squid3 (Ubuntu Trusty) Status: Fix Committed = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
I verified the fix with Oleg's test case (comment #16). I first reproduced the issue in a pristine trusty environment with squid3 (3.3.8-1ubuntu6.2) and then installed the proposed package (3.3.8-1ubuntu6.3) and got a TCP_MEM_HIT/200 just like Oleg's example. ** Tags removed: verification-needed ** Tags added: verification-done -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
Diogo will take a look for us to see if we can drive this through. ** Changed in: squid3 (Ubuntu Trusty) Assignee: Brian Murray (brian-murray) = Diogo Matsubara (matsubara) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1336742] Re: Caching responses with Vary header
On Wed, Jul 8, 2015 at 11:45 PM Chris J Arges 1336...@bugs.launchpad.net wrote: Hello Michael, or anyone else affected, Hi Chris, Unfortunately I'm not able to reproduce the original problem with the existing 3.3.8-1ubuntu6.2 (which is different to what I originally used) . As I mentioned in the description, I'd not tested with a pristine trusty environment, nor can I setup the same env with which I tested in the original description. In case I'm missing something obvious, and it helps someone else reproduce the original issue and/or confirm the fix, here is how I tried to reproduce the original problem just now on a freshly booted trusty instance: http://paste.ubuntu.com/11853981/ Of course, if we're unable to reproduce the original issue, it may have been specific to something in the environment I had (or have others seen the same?). One possibility is that the app server I was using was sending an incorrect Content-Length when Vary was present (as I noticed that an incorrect Content-Length header causes squid to not cache, which is understandable). -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
Hello Michael, or anyone else affected, Accepted squid3 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squid3/3.3.8-1ubuntu6.3 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: squid3 (Ubuntu Trusty) Status: In Progress = Fix Committed ** Tags added: verification-needed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
** Branch linked: lp:ubuntu/trusty-proposed/squid3 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
I'll go ahead and sponsor this. ** Changed in: squid3 (Ubuntu Trusty) Assignee: (unassigned) = Brian Murray (brian-murray) ** Changed in: squid3 (Ubuntu Trusty) Status: Triaged = In Progress -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
** Patch added: debdiff for trusty (untested) https://bugs.launchpad.net/ubuntu/trusty/+source/squid3/+bug/1336742/+attachment/4349019/+files/LP1336742-trusty.patch -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
This bug was fixed in the package squid3 - 3.3.8-1ubuntu12 --- squid3 (3.3.8-1ubuntu12) vivid; urgency=medium * debian/tests/testlib_httpd.py: Use service command instead of upstart specific ones, and simplify the logic. * debian/tests/testlib.py, check_exe(): Check /proc/pid/exe symlink instead of parsing cmdline; the latter has (squid-1) with the init.d script, and it's not really what we are interested in. -- Martin Pitt martin.p...@ubuntu.com Fri, 06 Mar 2015 12:10:59 +0100 ** Changed in: squid3 (Ubuntu) Status: Triaged = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
Updated package has been pushed to vivid-proposed. Feel free to try it. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
The attachment lp-1336742-vivid.debdiff seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the patch flag from the attachment, remove the patch tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
** Branch linked: lp:ubuntu/vivid-proposed/squid3 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
I prepared a debdiff for vivid (attached) and currently looking for sponsorship. My repro case can be found here: http://pastebin.ubuntu.com/10527246/ ** Patch added: lp-1336742-vivid.debdiff https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1336742/+attachment/4334428/+files/lp-1336742-vivid.debdiff -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1336742] Re: Caching responses with Vary header
On Fri, Feb 27, 2015 at 1:27 AM, Robie Basak 1336...@bugs.launchpad.net wrote: We're past feature freeze for Vivid now, so I can't just update to the latest any more, unless only bugfixes are involved or the release team grant an exception. If you can give me backported patches though, I can apply those. I backported the fix to trusty as per the attached branch. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
We're past feature freeze for Vivid now, so I can't just update to the latest any more, unless only bugfixes are involved or the release team grant an exception. If you can give me backported patches though, I can apply those. Otherwise a fix will need to wait until I can figure out backporting patches myself, or wait until Vivid is released. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to squid3 in Ubuntu. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
Debian Sid and Jesse package squid3 includes the fix for this bug. Please update the Ubuntu packages to match the upstream Debain repository. ** Also affects: squid3 (Debian) Importance: Undecided Status: New ** Changed in: squid3 (Debian) Status: New = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to squid3 in Ubuntu. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
** Bug watch added: Squid Bugzilla #3806 http://bugs.squid-cache.org/show_bug.cgi?id=3806 ** Also affects: squid via http://bugs.squid-cache.org/show_bug.cgi?id=3806 Importance: Unknown Status: Unknown ** Also affects: squid3 (Ubuntu Trusty) Importance: Undecided Status: New ** Changed in: squid3 (Ubuntu) Status: New = Triaged ** Changed in: squid3 (Ubuntu Trusty) Status: New = Triaged ** Changed in: squid3 (Ubuntu) Importance: Undecided = Medium ** Changed in: squid3 (Ubuntu Trusty) Importance: Undecided = Medium ** Tags added: bitesize -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to squid3 in Ubuntu. https://bugs.launchpad.net/bugs/1336742 Title: Caching responses with Vary header To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/1336742/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1336742] Re: Caching responses with Vary header
Launchpad has imported 12 comments from the remote bug at http://bugs.squid-cache.org/show_bug.cgi?id=3806. If you reply to an imported comment from within Launchpad, your comment will be sent to the remote bug automatically. Read more about Launchpad's inter-bugtracker facilities at https://help.launchpad.net/InterBugTracking. On 2013-03-13T19:50:48+00:00 Scott-harris wrote: When objects retrieved from web server return with a vary header, object is never cached and all future requests are always logged as a MISS. Disabling the vary header on web server causes the same object to be cached as expected. Seems to be same issues as discussed here : http://www.squid-cache.org /mail-archive/squid-dev/201207/0100.html Reply at: https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1336742/comments/0 On 2013-03-25T04:48:31+00:00 Amos Jeffries wrote: *** Bug 3799 has been marked as a duplicate of this bug. *** Reply at: https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1336742/comments/1 On 2013-03-25T04:54:29+00:00 Amos Jeffries wrote: Created attachment 2854 Set body size 0 on the vary marker object I am suspecting that the recent changes to cache max-size is related to this. The internal Vary marker object we use in the cache has no body but was specifying 'unknown' body length instead of none. Please test this patch. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1336742/comments/2 On 2013-03-26T10:39:50+00:00 Sbaynes wrote: (In reply to comment #2) Created attachment 2854 [details] Set body size 0 on the vary marker object I am suspecting that the recent changes to cache max-size is related to this. The internal Vary marker object we use in the cache has no body but was specifying 'unknown' body length instead of none. Please test this patch. Just tried the patch on Squid 3.2 in the context of bug 3799 and it did not make any difference. Does it require a higher version of Squid? Reply at: https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1336742/comments/3 On 2013-03-26T16:06:45+00:00 Scott-harris wrote: (In reply to comment #3) (In reply to comment #2) Created attachment 2854 [details] Set body size 0 on the vary marker object I am suspecting that the recent changes to cache max-size is related to this. The internal Vary marker object we use in the cache has no body but was specifying 'unknown' body length instead of none. Please test this patch. Just tried the patch on Squid 3.2 in the context of bug 3799 and it did not make any difference. Does it require a higher version of Squid? I also have tested with squid 3.3.3 and made no difference, what debugging can I turn on and logs to provide to help diagnose? Thanks Scott Reply at: https://bugs.launchpad.net/ubuntu/+source/squid3/+bug/1336742/comments/4 On 2013-04-04T00:35:00+00:00 Ahollyhock wrote: Please find my analysis on this issue. Squid 3.2.1 I have tracked this down to the way hash keys are generated and retrieved for requests with vary header. If you go to the squid source code file src/store_key_md5.cc (around line 100): const cache_key * storeKeyPublicByRequestMethod(HttpRequest * request, const HttpRequestMethod method) { static cache_key digest[SQUID_MD5_DIGEST_LENGTH]; unsigned char m = (unsigned char) method.id(); const char *url = urlCanonical(request); SquidMD5_CTX M; SquidMD5Init(M); SquidMD5Update(M, m, sizeof(m)); SquidMD5Update(M, (unsigned char *) url, strlen(url)); if (request-vary_headers) SquidMD5Update(M, (unsigned char *) request-vary_headers, strlen(request-vary_headers)); SquidMD5Final(digest, M); return digest; } you can see that MD5-based keys are generated based on request method id, request URL, and request vary header. The problem seems to be that when hash keys are generated during retrieval, the vary header field is empty and therefore although the resource is cached it cannot be retrieved because of the incorrect hash key. If you comment out two lines from above as follows: //if (request-vary_headers) //SquidMD5Update(M, (unsigned char *) request-vary_headers, strlen(request-vary_headers)); and recompile squid you will see that such resources get served from cache indicating that the culprit is the vary header. The part of the code that seems to be generating the hash keys for retrieval is inside src/store.cc (around line 625): StoreEntry * storeGetPublicByRequestMethod(HttpRequest * req, const HttpRequestMethod method) { return