Re: Backend content location
On 11/17/2013 11:27 PM, Viktor Villafuerte wrote: Hi all, I'd like to ask if it's possible to get the content location, from a pool of backend servers, instead of the content itself. Eg: backends are grouped by 'director hash' therefore each of them will have unique content. When my request is not found in the cache it then proceeds onto the backends and gets the content. I'd like to find the location such as server1.domain.com/content1 instead of actually getting the content. Is this possible? v..v Would set beresp.http.be_name = beresp.backend.name; Be what you are looking for? (taken from bin/varnishtest/tests/v00028.vtc) -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Multiple backends on same server (cPanel)
On 04/07/2014 04:26 PM, Jerry Steele wrote: > set.req.backend = default; Try: set req.backend = default; (no dot) -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish reduces hit rate
On 04/23/2014 10:13 AM, Joydeep Bakshi wrote: > Dear list, > > When use nginx+apache the web hit rates maximize dramatically. But > when introduce varnish in between as nginx->varnish->apache the hit > rates reduce. > How to optimize varnish to get higher hit rates ? > Did you give the same storage space to varnish as you did to nginx? This can be of interest: https://www.varnish-cache.org/docs/3.0/tutorial/increasing_your_hitrate.html -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish reduces hit rate
On 04/23/2014 10:30 AM, Joydeep Bakshi wrote: > sorry but I failed to understand "same storage space". Even I have > not done any such configuration tewak. > > Though I like to share first few lines of varnishstat which shows the > poor cache_hit > > 0+00:57:24 > Hitrate ratio:222 > Hitrate avg: 0.0149 0.0149 0.0149 > >35062 0.0010.18 client_conn - Client > connections accepted >35062 0.0010.18 client_req - Client requests > received >2 0.00 0.00 cache_hit - Cache hits >34994 0.0010.16 cache_hitpass - Cache hits for pass > 66 0.00 0.02 cache_miss - Cache misses >34975 0.0010.16 backend_conn - Backend conn. > success > You have a high hitpass ratio, your backend may not be sending TTL/Age information, causing varnish to avoid caching. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish reduces hit rate
On 04/23/2014 10:57 AM, Joydeep Bakshi wrote: > can we do something to improve ? > You can force the TTL to something positive if it's <= 0s on the varnish side. And on the backend side, you can configure the server to send the ttl headers. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish reduces hit rate
On 04/23/2014 11:55 AM, Joydeep Bakshi wrote: > Since I am completely new to varnish, may I request to share the > knowledge doing this ? I recommend reading the documentation first, alot of your answers are in them. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish mp4 streaming
On 06/10/2014 03:07 AM, Hernán Marsili wrote:
> Hi,
>
> We are working on a VIDEO SITE for a sports sites. Currently working
> with Varnish 3.0.5. The average video size is 100mb. We have a 4gb
> malloc storage.
>
> Right now, we are handling mp4 as just a regular file. No problems so
> far. A couple of questions:
>
> 1) STREAM. We tried this on the vcl but we don't see much improvement.
> Should we use it anyway?
>
> if (req.url ~ "\.mp4") {
>
>set beresp.do_*stream* = true;
>
>set beresp.http.X-Cacheable-TTL = "stream";
>
> }
>
>
> 1) VARNISH STREAM BRANCH. We find a all branch of varnish for
> streaming. https://github.com/mbgrydeland/varnish-cache-streaming.
> Does anyway knows if this is now part of Varnish 3.0.5 or Varnish 4?
>
>
You may have two problems:
- the fact that your file is being downloaded, meaning that only the
client who triggered the backend request will profit from the streaming.
This is solved in v4, and data will be send as soon as available, for
all threads.
- the video metadata is at the end of the file, meaning that your
clients must retrieve all the file before starting to read. You're
probably in this case if the video takes a long time to read, even if
cached.
To mitigate the second problem, you may use qt-faststart
(https://github.com/danielgtaylor/qtfaststart) to move the meta data at
the beginning of the file, or switch to a chunked protocol.
(hopefully, if per-request vmods calls come to fruition, or the vfp
mature a bit more, we'll have a vmod to qt-faststartify files on the fly).
Hope that helps.
--
Guillaume Quintard
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: GeoIP Detection and Country Specific Content
On 01/29/2015 04:35 PM, Max Clark wrote: > Hello, > > We have been working to enable region specific content for our web > site. The basic mechanism is already in place, specifically: > > When a user visits the site and a cookie is not present the > application server will compare their IP address with the GeoIP > database. Based on the result the application server will generate the > proper content for the user (different logo images, text, etc...) and > return this with a cookie specifying the edition for future requests > bypassing the GeoIP lookup. The cookie will also allow the user to > select a different region than what their IP resolves to. > > Again this is working in the application, and Varnish was extended to > match on the cookie value (named "edition") and to use the edition > value as part of the hash data so there is a unique cache for the > different regions. > > Here's the problem: > > When a user visits the site for the first time they do not have a > cookie in the request. So Varnish has a blank value for the edition as > part of the hash data. However this request when it hits the > application server does resolve to a specific edition and return the > region content on the reply. This is being cached by Varnish and used > for future requests by new users with no cookie. > > Put simply, if the page is not in cache, and a browser from the > UK does not have the edition cookie a page with UK content is > generated and cached in Varnish. If a browser from the US goes to the > same page, Varnish will serve the US user the UK cached page because > this is what was cached based on the hash data. > > Without turning off caching for cookieless requests - how do we > resolve this? > I may not have understood exactly what you need, but can't you hash the country code instead of the cookie ? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish purge behind nginX proxy
On 04/15/2015 04:44 PM, Bertrand Caplet wrote: > Because of SSL termination. > Purging through commandline would be fine to me too. But I couldn't > find a working command for 4.0 that flush all site. > > Regards, You'll probably have to fill a header (X-Forwarded-for is an obvious candidate) with nginx and check against that in varnish. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish purge behind nginX proxy
On 04/16/2015 07:42 PM, Bertrand Caplet wrote: > To make it clear for you here's my config: > > > Client ---> Nginx Reverse SSL ---> Varnish ---> Nginx + PHP-FPM > > X-Forwarded-for is already completed by nginx (and working ofc) and I > tried as Per Buer told me with this rule : > > if (std.ip(req.http.X-forwarded-for, "0.0.0.0") !~ purge) > > but I got the following error : > > Message from VCC-compiler: > Symbol not found: 'std.ip' (expected type BOOL) > could you be missing a "import std;" ? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Conditional backend requests (ETag)
Hi Magnus, The documention you are looking for is here : https://www.varnish-cache.org/docs/4.0/reference/vcl.html And beresp.keep is definitely what you are looking for. If ttl is expired but not ttl + keep, varnish will issue a conditionnal request. -- Guillaume Quintard On Fri, Sep 18, 2015 at 12:35 PM, Magnus Hoff wrote: > Hi :) > > What is the status of support for conditional backend requests? > > My use case is accelerating CouchDB, which has good support for ETags, > but offers no other caching opportunities. > > I have found [1] which leads me to think that the branch implementing > this feature was merged to trunk in 2011. According to [2] (which also > says, in big bold letters, "This documentation is out of date"), I > should be able to see this working with the default VCL: "In the > default case (i.e. if not superseded by something in VCL), a > conditional request is generated during fetch with If-Modified-Since > and/or If-None-Match headers whose values are taken from stale_obj's > Last-Modified and/or ETag headers." > > [1]: > https://www.varnish-cache.org/lists/pipermail/varnish-commit/2011-July/006720.html > [2]: https://www.varnish-cache.org/trac/wiki/BackendConditionalRequests > > However, when testing with the latest Varnish from the Varnish Ubuntu > repository (https://www.varnish-cache.org/installation/ubuntu), > "varnish-4.0.3 revision b8c4a34", I am unable to trigger conditional > backend requests in the default configuration. I have feebly tried > making the VCL more explicit in that it should try to keep the ETagged > versions around by setting beresp.keep explicitly, but I can't seem to > effect a change in behavior. > > So, what is happening? Does Varnish currently support what I am > looking for? What is the up to date documentation on this? Is there > support for this on a branch still? How is that branch coming along? > > > Thank you :) > - Magnus Hoff > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Caching authenticated responses
Hi Anastasios, This is totally feasible, look at the vcl_hash fonction in docs, it'll allow to control what goes into the hash. On Oct 4, 2015 21:41, "Anastasios Papadopoulos" wrote: > Is there some easy way to configure Varnish to cache authenticated > responses? > The concept is quite simple: the back-end is a REST API provider and the > response payload is mostly cacheable (i.e. rarely changes). The API uses > basic HTTP authentication which prevents default Varnish to cache it. > However, since the user base is limited, I'd like Varnish to cache > (meaning: respect the back-end caching headers, not blindly cache anything) > using the authorization header as part of the hash / cache key. Is this > even possible? > Any sample VCL or pointers would be welcomed! > > Thank you very much! > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Graphing number of 503s
Hi Gabriel, Since the vsl is a circular buffer, your are forced to watch for the time period you are interested in. But it's probably easier to just use varnishncsa for what you want instead of varnishlog. On Oct 15, 2015 06:34, "Gabriel Filion" wrote: > Hello there, > > I'd like to create a graph of how much 503 pages were served by varnish. > Is there an easy way to count how much such error pages are sent to > clients? > > The only way that I can think right now is to have a permanent > varnishlog that is set to spot the 503s and then use the output from > varnishlog to create the numbers. > > I was wondering if there might be a more clever way of doing this. > > -- > Gabriel Filion > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Purging behaviour
On Dec 5, 2015 15:07, "Paulo Silva" wrote: > Since the purge requests will only have the hostname and url, is purge smart enough to remove from cache both versions or will it remove only the object without the cookie? If only one version is removed is there some way to force varnish to remove both versions (without issuing a second PURGE request with the cookie)? Purge is not smart :-) it will purge a whole Objhead, ie. all objects with the same hash. Unfortunately, what you are doing stores the objects under different hashes. What you need here is the vary-header. I won't go into details now (you should have plenty of info in the varnish book), but basically, you can set req.http.mycookie = "yes" or "no" and beresp.http.vary = "mycookie" and varnish will do what you expect. ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Why varnish caches as the cache is stopped?
On Dec 12, 2015 11:36, "georgi@gmail.com" wrote:
>
> Hello,
> I want to use varnish only as a proxy server, not to cache and have the
following in the configuration which should stop the cache completely in
vcl recv:
>
> if (req.http.Authorization || req.http.Cookie) {
> return (pass);
> }
> }
>
> the age header is 0, so it should not cache, but I notice that
varnish_storage.bin is full although, so something is cached - right? On
some servers with 32 GB ram29 is full and when I restart varnish the ram is
no more used, but I am not sure it these two things are linked. Here is my
/etc/sysconfig/varnish:
>
> # # The minimum number of worker threads to start
> VARNISH_MIN_THREADS=50
>
> # # The Maximum number of worker threads to start
> VARNISH_MAX_THREADS=1000
>
> # # Idle timeout for worker threads
> VARNISH_THREAD_TIMEOUT=120
>
> # # Cache file location
> VARNISH_STORAGE_FILE=/usr/local/varnish/lib/varnish_storage.bin
>
> # # Cache file size: in bytes, optionally using k / M / G / T suffix,
> # # or in percentage of available disk space using the % suffix.
> VARNISH_STORAGE_SIZE=1G
>
> # # Backend storage specification
> VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
>
> # # Default TTL used when the backend does not specify one
> VARNISH_TTL=120
>
> # # Varnish username
> VARNISHUSER=varnish
>
> # # Varnish group name
> VARNISHGROUP=varnish
>
> DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
> -T
${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
> -f ${VARNISH_VCL_CONF} \
> -S ${VARNISH_SECRET_FILE} \
> -s file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
>
>
>
> My questions are 1) does my conviguration stop completely the cache and
why then storage.bin is full?
> 2) Why varnish eat so much ram - because of many requests, wrong
configuration or?
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Hi,
The best way to investigate here is to look at the varnishlog and check if
indeed you end up in hit or miss instead of pass.
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Why varnish caches as the cache is stopped?
On Dec 14, 2015 17:47, "georgi@gmail.com" wrote:
>
> Logs didn't help me. Now I faced the same problem with php file which
didn't refresh until varnish restart. After I put the following in vcl_recv
{ the problem was solved:
>
> if (req.url ~
"\.(png|gif|jp(e?)g|avi|flv|mp(e?)g|mp4|mp3|php|js)"){return(pass);}
>
> But why I should add this after I have :
>
>
> if (req.http.Authorization || req.http.Cookie) {
> return (pass);
> }
> }
>
> and is there better solution than the first one that wills top
completely the cache? Also, I use the same config file on all servers and
there isn't such a problem now, only on this customer.
>
> This is the log of the request befor I add the pass for certaing type of
files:
>
>58 TxResponse c OK
>58 TxHeader c Server: Apache
>58 TxHeader c Last-Modified: Mon, 12 Jul 2010 06:40:00 GMT
>58 TxHeader c Content-Type: text/html
>58 TxHeader c Content-Length: 5774
>58 TxHeader c Accept-Ranges: bytes
>58 TxHeader c Date: Mon, 14 Dec 2015 16:30:58 GMT
>58 TxHeader c Connection: keep-alive
>58 Length c 5774
>58 ReqEnd c 2078285659 1450110658.014697075 1450110658.033298254
1.481907368 0.018534422 0.66757
>69 Debugc herding
>69 SessionClose c timeout
>69 StatSess c 212.233.149.2 54693 3 1 14 0 14 14 3823 211709
>16 Debugc herding
>16 Interrupted c ReqStart
>16 ReqStart c 1.1.1.1 33259 2078285660
>16 RxRequestc GET
>16 RxURLc /testov.php
>16 RxProtocol c HTTP/1.1
>16 RxHeader c Host: siteofmycustomer.com
>16 RxHeader c User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686;
rv:42.0) Gecko/20100101 Firefox/42.0
>16 RxHeader c Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>16 RxHeader c Accept-Language: en-US,en;q=0.5
>16 RxHeader c Accept-Encoding: gzip, deflate
>16 RxHeader c Connection: keep-alive
>16 RxHeader c Cache-Control: max-age=0
>16 VCL_call c recv lookup
>16 VCL_call c hash
>16 Hash c /testov.php
>16 Hash c termoproekt.bg
>16 VCL_return c hash
>16 Hit c 2078285077
>16 VCL_call c hit deliver
>16 VCL_call c deliver deliver
>16 TxProtocol c HTTP/1.1
>16 TxStatus c 200
>16 TxResponse c OK
>16 TxHeader c Server: Apache
>16 TxHeader c Content-Type: text/html; charset=UTF-8
>16 TxHeader c Content-Length: 0
>16 TxHeader c Accept-Ranges: bytes
>16 TxHeader c Date: Mon, 14 Dec 2015 16:30:58 GMT
>16 TxHeader c Connection: keep-alive
>16 Length c 0
>
Looking at the request (id 16), I see no reason for it to pass as it has no
cookie or authorization header.
Varnish seems to be doing the right thing.
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Why varnish caches as the cache is stopped?
Why wouldn't it be cached? Sorry but I'm not getting the problem here. -- Guillaume Quintard On Mon, Dec 14, 2015 at 6:56 PM, georgi@gmail.com wrote: > Yes, but the result were cached and can't understand why. While it's > cached on port 80 where the varnish is, opening the page with :8080 was ok > (on the apache side). Any other suggestions? > > > On 12/14/2015 07:04 PM, Guillaume Quintard wrote: > >> Server: Apache >> >16 TxHeader c Content-Type: text/html; charset=UTF-8 >> >16 TxHeader c Content-Length: 0 >> >16 TxHeader c Accept-Ranges: by >> > > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Bypass cache when add to cart ?
On Dec 23, 2015 20:51, "Xavier Cardil Coll" wrote:
>
> No one knows then ? Not even a hint or something ?
>
> 2015-12-23 13:15 GMT+01:00 Xavier Cardil Coll :
>>
>> Also, how to write a function to bypass cache entirely when a user logs
in ?
>>
>> 2015-12-23 12:39 GMT+01:00 Xavier Cardil Coll :
>>>
>>> On Nginx cache, you can do this to avoid keep on caching when you add
something to the cart in Woocommerce :
>>>
>>> Skip cache for WooCommerce query string
>>>
>>> if ( $arg_add-to-cart != "" ) {
>>> set $skip_cache 1;
>>> }
>>>
>>> Skip cache when WooCommerce cart is not empty
>>>
>>> if ( $cookie_woocommerce_items_in_cart != "0" ) {
>>> set $skip_cache 1;
>>> }
>>>
>>> How can you do this in Varnish ? With this config, wich was made
specifically for Wordpress + Woocommerce, when you add something to cart,
cache is not disable store wide so you get a mix of cached and non cache
pages, cart gets emptied : http://pastebin.com/TuYeMtUZ
>>>
>>>
>>> --
>>>
>>> ELSITAR
>>>
>>>
>>
>>
>>
>> --
>>
>> ELSITAR
>>
>>
>
>
>
> --
>
> ELSITAR
>
>
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Chill out, most of the people are in vacation :-)
So, in varnish, by default, if the request contains a cookie, the cache is
bypassed.
But you can also force a pass, for example if the request contains a
"foo-bar" header like so :
sub vcl_recv {
if (req.http.foo-bar) {
return (pass);
}
}
For your needs, look at the query string and cookie vmods, it will allow to
really fine-tune your workflow.
And should you need more help, we are here :-)
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: using http/2 with varnish
Hi, For a more detailed answer : we don't support H/2 in varnish yet (working on it!). So, if you really really want H/2, having nginx in front of varnish can be a solution. If you are only interested in https, however, varnish 4.1 and onward supports the proxy protocol. It will allow to use and SSL/TLS terminator such as hitch or haproxy that will handle the encryption for you. The advantage to using the proxy protocol is that varnish is aware of it. If you use nginx to proxy the requests, varnish will only see one client: nginx. This means you'll have to do some gymnastics with XFF headers if you want to filter by ip address for example. Plus, nginx is a bit overkill in terms of resources to just be a tls terminator. Migrating to varnish 4 requires a bit of work (not that much, really), but it's worth it, especially considering v3 is EOL. -- Guillaume Quintard On Sun, Dec 27, 2015 at 9:27 PM, Mattias Geniar wrote: > > Can anyone point us on the right direction here? > > What you need is a reverse proxy in front of your Varnish instances: > consider running a tool like Nginx (which has HTTP/2 support in its > mainline repositories) that does all your TLS connections and proxies the > request on to Varnish, to keep optimising the cache. > > Mattias > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: why i get response if it is a post request with pass?
It looks like your backend failed to deliver ("http first read error", then
"backend write error"), so you were taken to vcl_backend_error then to
vcl_deliver. Could it be that your vcl has a vcl_backend_error that
rewrites the error code?
>From the varnish side, your backend failed, it may be a timeout problem
(backend replied too late).
--
Guillaume Quintard
On Sun, Dec 27, 2015 at 12:32 PM, יעקב ירמולובסקי
wrote:
> I checked my apache log and it responded with 200.
>
> 14 BackendXID b 2005611725
>14 TxRequestb POST
>14 TxURLb /node/16348/edit?destination=user/18407
>14 TxProtocol b HTTP/1.1
>14 TxHeader b Host: www.limudnaim.co.il
>14 TxHeader b Content-Length: 7411
>14 TxHeader b Cache-Control: max-age=0
>14 TxHeader b Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
>14 TxHeader b Origin: http://www.limudnaim.co.il
>14 TxHeader b Upgrade-Insecure-Requests: 1
>14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
>14 TxHeader b Content-Type: multipart/form-data;
> boundary=WebKitFormBoundaryHUWnuy0mKyJkVjht
>14 TxHeader b Referer:
> http://www.limudnaim.co.il/node/16348/edit?destination=user/18407
>14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4
>14 TxHeader b X-UA-Device: pc
>14 TxHeader b X-Full-Uri:
> www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407
>14 TxHeader b X-Forwarded-For: 46.121.61.13
>14 TxHeader b Accept-Encoding: gzip
>14 TxHeader b Cookie:
> SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9;
> NO_CACHE=Y
>14 TxHeader b X-Varnish: 2005611725
>14 BackendClose b default
>14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012
>14 BackendXID b 2005611725
>14 TxRequestb POST
>14 TxURLb /node/16348/edit?destination=user/18407
>14 TxProtocol b HTTP/1.1
>14 TxHeader b Host: www.limudnaim.co.il
>14 TxHeader b Content-Length: 7411
>14 TxHeader b Cache-Control: max-age=0
>14 TxHeader b Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
>14 TxHeader b Origin: http://www.limudnaim.co.il
>14 TxHeader b Upgrade-Insecure-Requests: 1
>14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
>14 TxHeader b Content-Type: multipart/form-data;
> boundary=WebKitFormBoundaryHUWnuy0mKyJkVjht
>14 TxHeader b Referer:
> http://www.limudnaim.co.il/node/16348/edit?destination=user/18407
>14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4
>14 TxHeader b X-UA-Device: pc
>14 TxHeader b X-Full-Uri:
> www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407
>14 TxHeader b X-Forwarded-For: 46.121.61.13
>14 TxHeader b Accept-Encoding: gzip
>14 TxHeader b Cookie:
> SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9;
> NO_CACHE=Y
>14 TxHeader b X-Varnish: 2005611725
>14 BackendClose b default
>17 SessionOpen c 46.121.61.13 4316 :80
>17 ReqStart c 46.121.61.13 4316 2005611725
>17 RxRequestc POST
>17 RxURLc /node/16348/edit?destination=user/18407
>17 RxProtocol c HTTP/1.1
>17 RxHeader c Host: www.limudnaim.co.il
>17 RxHeader c Connection: keep-alive
>17 RxHeader c Content-Length: 7411
>17 RxHeader c Cache-Control: max-age=0
>17 RxHeader c Accept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
>17 RxHeader c Origin: http://www.limudnaim.co.il
>17 RxHeader c Upgrade-Insecure-Requests: 1
>17 RxHeader c User-Agent: Mozilla/5.0 (Windows NT 5.1)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
>17 RxHeader c Content-Type: multipart/form-data;
> boundary=WebKitFormBoundaryHUWnuy0mKyJkVjht
>17 RxHeader c Referer:
> http://www.limudnaim.co.il/node/16348/edit?destination=user/18407
>17 RxHeader c Accept-Encoding: gzip, deflate
>17 RxHeader c Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4
>17 RxHeader c Cookie:
> SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9;
> __utmt=1; NO_CACHE=Y;
> __utma=77876233.313860119.1451212742.1451212742.1451212742.1;
> __utmb=77876233.22.9.1451213179574; __utmc=77876233;
> __utmz=77876233.1451212742.1.1.utmcsr=goog
Re: why i get response if it is a post request with pass?
Well, duh. Sorry, I didn't even notice you were on v3.
How much time does Apache take to answer? It might be just a tiny bit more
than what varnish would like.
Do you have a custom vcl_error?
--
Guillaume Quintard
On Mon, Dec 28, 2015 at 10:25 AM, יעקב ירמולובסקי
wrote:
> I'm using varnish 3 so I don't think that I have vcl_backend_error.
> As for the backend, I'm using Apache 2.2.31 and I have checked the logs
> but it seems that everything goes well there and I get status 200.
> Maybe I need to enable something in the Apache config?
>
> On Mon, Dec 28, 2015 at 10:55 AM, Guillaume Quintard <
> guilla...@varnish-software.com> wrote:
>
>> It looks like your backend failed to deliver ("http first read error",
>> then "backend write error"), so you were taken to vcl_backend_error then
>> to vcl_deliver. Could it be that your vcl has a vcl_backend_error that
>> rewrites the error code?
>>
>> From the varnish side, your backend failed, it may be a timeout problem
>> (backend replied too late).
>>
>> --
>> Guillaume Quintard
>>
>> On Sun, Dec 27, 2015 at 12:32 PM, יעקב ירמולובסקי
>> wrote:
>>
>>> I checked my apache log and it responded with 200.
>>>
>>> 14 BackendXID b 2005611725
>>>14 TxRequestb POST
>>>14 TxURLb /node/16348/edit?destination=user/18407
>>>14 TxProtocol b HTTP/1.1
>>>14 TxHeader b Host: www.limudnaim.co.il
>>>14 TxHeader b Content-Length: 7411
>>>14 TxHeader b Cache-Control: max-age=0
>>>14 TxHeader b Accept:
>>> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
>>>14 TxHeader b Origin: http://www.limudnaim.co.il
>>>14 TxHeader b Upgrade-Insecure-Requests: 1
>>>14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1)
>>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
>>>14 TxHeader b Content-Type: multipart/form-data;
>>> boundary=WebKitFormBoundaryHUWnuy0mKyJkVjht
>>>14 TxHeader b Referer:
>>> http://www.limudnaim.co.il/node/16348/edit?destination=user/18407
>>>14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4
>>>14 TxHeader b X-UA-Device: pc
>>>14 TxHeader b X-Full-Uri:
>>> www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407
>>>14 TxHeader b X-Forwarded-For: 46.121.61.13
>>>14 TxHeader b Accept-Encoding: gzip
>>>14 TxHeader b Cookie:
>>> SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9;
>>> NO_CACHE=Y
>>>14 TxHeader b X-Varnish: 2005611725
>>>14 BackendClose b default
>>>14 BackendOpen b default 82.80.17.31 39482 82.80.17.31 1012
>>>14 BackendXID b 2005611725
>>>14 TxRequestb POST
>>>14 TxURLb /node/16348/edit?destination=user/18407
>>>14 TxProtocol b HTTP/1.1
>>>14 TxHeader b Host: www.limudnaim.co.il
>>>14 TxHeader b Content-Length: 7411
>>>14 TxHeader b Cache-Control: max-age=0
>>>14 TxHeader b Accept:
>>> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
>>>14 TxHeader b Origin: http://www.limudnaim.co.il
>>>14 TxHeader b Upgrade-Insecure-Requests: 1
>>>14 TxHeader b User-Agent: Mozilla/5.0 (Windows NT 5.1)
>>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36
>>>14 TxHeader b Content-Type: multipart/form-data;
>>> boundary=WebKitFormBoundaryHUWnuy0mKyJkVjht
>>>14 TxHeader b Referer:
>>> http://www.limudnaim.co.il/node/16348/edit?destination=user/18407
>>>14 TxHeader b Accept-Language: he-IL,he;q=0.8,en-US;q=0.6,en;q=0.4
>>>14 TxHeader b X-UA-Device: pc
>>>14 TxHeader b X-Full-Uri:
>>> www.limudnaim.co.il%2Fnode%2F16348%2Fedit%3Fdestination%3Duser%2F18407
>>>14 TxHeader b X-Forwarded-For: 46.121.61.13
>>>14 TxHeader b Accept-Encoding: gzip
>>>14 TxHeader b Cookie:
>>> SESSdac9e0acd970131ed940ca8f937850da=cc448a1c9de0973a7878d6e0941536c9;
>>> NO_CACHE=Y
>>>14 TxHeader b X-Varnish: 2005611725
>>>14 BackendClose b default
>>>17 SessionOpen c 46.121.61.13 4316 :80
>>>17 ReqStart c 46.121.61.13 4316 2005611725
>>>17 RxRequestc POST
>>>17 RxURL
Re: using http/2 with varnish
I think our goal is mid 2016, hopefully sooner, but don't take that as a promise. -- Guillaume Quintard On Mon, Dec 28, 2015 at 12:29 PM, Hernán Marsili wrote: > Thank you very much. Is there an estimate time for h2 support? (first > quarter 2016, mid-year, etc?). Just to have a notion. > > Best regards, > Hernán. > > > On Mon, Dec 28, 2015 at 5:48 AM Guillaume Quintard < > guilla...@varnish-software.com> wrote: > >> Hi, >> >> For a more detailed answer : we don't support H/2 in varnish yet (working >> on it!). So, if you really really want H/2, having nginx in front of >> varnish can be a solution. >> >> If you are only interested in https, however, varnish 4.1 and onward >> supports the proxy protocol. It will allow to use and SSL/TLS terminator >> such as hitch or haproxy that will handle the encryption for you. >> >> The advantage to using the proxy protocol is that varnish is aware of it. >> If you use nginx to proxy the requests, varnish will only see one client: >> nginx. This means you'll have to do some gymnastics with XFF headers if you >> want to filter by ip address for example. Plus, nginx is a bit overkill in >> terms of resources to just be a tls terminator. >> >> Migrating to varnish 4 requires a bit of work (not that much, really), >> but it's worth it, especially considering v3 is EOL. >> >> -- >> Guillaume Quintard >> >> On Sun, Dec 27, 2015 at 9:27 PM, Mattias Geniar >> wrote: >> >>> > Can anyone point us on the right direction here? >>> >>> What you need is a reverse proxy in front of your Varnish instances: >>> consider running a tool like Nginx (which has HTTP/2 support in its >>> mainline repositories) that does all your TLS connections and proxies the >>> request on to Varnish, to keep optimising the cache. >>> >>> Mattias >>> >> ___ >>> varnish-misc mailing list >>> varnish-misc@varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Installing Varnis on Debian 7
Paul is onto something here (I'm on arch, but I still have some vague
debian memories). You should be able to:
apt-get build-dep varnish
apt-get source varnish
cd varnish
dpkg-buildpackage
Or something similar.
--
Guillaume Quintard
On Tue, Dec 29, 2015 at 11:12 AM, Paul A. Procacci
wrote:
> On Tue, Dec 29, 2015 at 11:59:20AM +0530, Debraj Manna wrote:
> >Hi,
> >I am trying to install Varnish 4 on Debian 7 as mentioned in this
> >[1]link.
> >But on doing apt-get install varnish. I am getting the below error.
> >root@SPK-D-0476:/home/debraj# apt-get install varnish
> >Reading package lists... Done
> >Building dependency tree
> >Reading state information... Done
> >Some packages could not be installed. This may mean that you have
> >requested an impossible situation or if you are using the unstable
> >distribution that some required packages have not yet been created
> >or been moved out of Incoming.
> >The following information may help to resolve the situation:
> >The following packages have unmet dependencies:
> > varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is to be
> >installed
> > Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is to be
> >installed
> > Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not
> >going to be installed
> >E: Unable to correct problems, you have held broken packages.
> >I also tried aptitude but it is showing me the option to install
> >varnish 3 but not varnish 4
> >root@SPK-D-0476:/home/debraj# aptitude install varnish
> >The following NEW packages will be installed:
> > varnish{b}
> >0 packages upgraded, 1 newly installed, 0 to remove and 40 not
> >upgraded.
> >Need to get 527 kB of archives. After unpacking 1,636 kB will be used.
> >The following packages have unmet dependencies:
> > varnish : Depends: libc6 (>= 2.14) but 2.13-38+deb7u8 is installed.
> > Depends: libjemalloc1 (>= 2.1.1) but it is not going to be
> >installed.
> > Depends: libpcre3 (>= 1:8.35) but 1:8.30-5 is installed.
> > Depends: libvarnishapi1 (>= 4.1.0-1~jessie) but it is not
> >going to be installed.
> >The following actions will resolve these dependencies:
> > Keep the following packages at their current version:
> >1) varnish [Not Installed]
> >Accept this solution? [Y/n/q/?] n
> >The following actions will resolve these dependencies:
> > Install the following packages:
> >1) libvarnishapi1 [3.0.2-2+deb7u1 (oldstable)]
> >2) varnish [3.0.2-2+deb7u1 (oldstable)]
> >Accept this solution? [Y/n/q/?]
> >Can some one let me know how can I install varnish 4 on Debian 7?
> >Thanks,
> >
>
> Not sure about Debian, I'm a FreeBSD user myself, but when all else fails
> you
> can download the source and build your own rpm against the libs installed
> on
> your own system.
>
> ~Paul
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish 4.1 to HTTP/S backend
Hi Phil, It's not :-) Using the PROXY protocol, clients can talk using HTTP/S to your varnish box, but HTTP/S to backend is a Varnish Cache Plus exclusive for now. -- Guillaume Quintard On Fri, Jan 8, 2016 at 9:46 AM, Phil Daws wrote: > Hello: > > I read that Varnish 4.1 is now able to pull from a HTTP/S backend but > could not find any examples of how to do that; is that correct ? > > Thanks, Phil > > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish caching and the X-Requested-With header
Hi Angelo, Your first step should be to look at the varnishlog, find the relevant request, and compare with what your vcl (and the builtin one) to understand the behaviour. -- Guillaume Quintard On Thu, Jan 14, 2016 at 8:55 PM, Angelo Höngens wrote: > Hey, I'm trying to debug varnish not caching some request. I see the > uncacheable object has the request header X-Requested-With: XMLHttpRequest, > and I guess that somehow make Varnish not cache the request. But I cannot > find this anywhere in the documentation. Is this expected behavior? In > which module (vcl_recv, vcl_deliver) can I undo this behavior? > > -- > > With kind regards, > > Angelo Höngens > Systems Administrator > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Conditional requests / if-modified-since not working
Hi Raphaël,
Varnish ignores the must revalidate header, if you wish to emulate the
behaviour, you can use something like this in your vcl:
sub vcl_backend_response {
if (beresp.http.cache-control ~ "must-revalidate") {
set beresp.ttl = 1s;
set beresp.grace = 0s;
set beresp.keep = 1w;
}
}
This way, you'd only cache for 1 second (don't set it to 0, or all the
requests for this object would be done sequentially), but will keep the
object for a week, revalidating it each time it is requested and its ttl is
expired.
--
Guillaume Quintard
On Fri, Jan 22, 2016 at 3:32 PM, Raphaël Malié
wrote:
> Hello,
>
> I did a fresh install of Varnish 4.1 to test conditional requests with a
> last-modified header sent from the backend, here are my backend headers:
>
> Cache-Control: must-revalidate, proxy-revalidate, public
> Last-Modified: Fri, 22 Jan 2016 14:27:50 GMT
>
> On the next query, Varnish should try to contact the backend with a
> if-modified-since header, and the backend shoud send back a "304 not
> modified" answer if nothing changed.
>
> But Varnish never tries to contact the backend, he is always caching my
> page with the default ttl (120s).
>
> Thank you for any help,
> Raphaël
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Conditional requests / if-modified-since not working
You are welcome. Have fun.
--
Guillaume Quintard
On Sat, Jan 23, 2016 at 12:14 AM, Raphaël Malié
wrote:
> Hi Guillaume,
>
> It works, finally!
>
> Many thanks,
> Raphaël
>
> 2016-01-22 21:50 GMT+01:00 Guillaume Quintard <
> guilla...@varnish-software.com>:
>
>> Hi Raphaël,
>>
>> Varnish ignores the must revalidate header, if you wish to emulate the
>> behaviour, you can use something like this in your vcl:
>>
>> sub vcl_backend_response {
>> if (beresp.http.cache-control ~ "must-revalidate") {
>> set beresp.ttl = 1s;
>> set beresp.grace = 0s;
>> set beresp.keep = 1w;
>> }
>> }
>>
>> This way, you'd only cache for 1 second (don't set it to 0, or all the
>> requests for this object would be done sequentially), but will keep the
>> object for a week, revalidating it each time it is requested and its ttl is
>> expired.
>>
>> --
>> Guillaume Quintard
>>
>> On Fri, Jan 22, 2016 at 3:32 PM, Raphaël Malié
>> wrote:
>>
>>> Hello,
>>>
>>> I did a fresh install of Varnish 4.1 to test conditional requests with a
>>> last-modified header sent from the backend, here are my backend headers:
>>>
>>> Cache-Control: must-revalidate, proxy-revalidate, public
>>> Last-Modified: Fri, 22 Jan 2016 14:27:50 GMT
>>>
>>> On the next query, Varnish should try to contact the backend with a
>>> if-modified-since header, and the backend shoud send back a "304 not
>>> modified" answer if nothing changed.
>>>
>>> But Varnish never tries to contact the backend, he is always caching my
>>> page with the default ttl (120s).
>>>
>>> Thank you for any help,
>>> Raphaël
>>>
>>> ___
>>> varnish-misc mailing list
>>> varnish-misc@varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>
>>
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Catch /js on all domains and rewrite the host
Hi Matt,
would something like that do:
sub vcl_recv {
if (req.url ~ "^/js/") {
set req.http.host = "subdomain.domain.tld";
}
}
?
--
Guillaume Quintard
On Sun, Jan 31, 2016 at 6:31 PM, Matt . wrote:
> Hi All,
>
> I'm fuguring out how I can accomplish the following:
>
> I have subdomain.domain.tld but as this is a rewrite the olders like
> /js are not directly available on it.
>
> As my subdomains .domain.tld are dynamic I wonder if I can catch /js
> from reg.url for all incoming host(requests) and rewrite it to a
> source folder on another host so it can be served on:
>
> subdomain.domain.tld/js
>
> My script lookups /js/whatever.js which is needed for these libs.
>
> Is there some catchall technique to accomplish this ?
>
> Thanks!
>
> Matt
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Difference between n_object, n_objectcore & n_objecthead
>From bin/varnishd/cache/cache_hash.c: * We have two kinds of structures, objecthead and object. An objecthead * corresponds to a given (Host:, URL) tupple, and the objects hung from * the objecthead may represent various variations (ie: Vary: header, * different TTL etc) instances of that web-entity. And from bin/varnishd/cache/cache.h: /* Object core structure - * Objects have sideways references in the binary heap and the LRU list * and we want to avoid paging in a lot of objects just to move them up * or down the binheap or to move a unrelated object on the LRU list. * To avoid this we use a proxy object, objcore, to hold the relevant * housekeeping fields parts of an object. */ -- Guillaume Quintard On Fri, Jan 22, 2016 at 8:41 PM, Debraj Manna wrote: > Can someone point me to some doc where these stats are explained? > On Jan 22, 2016 5:37 PM, "Debraj Manna" wrote: > >> Hi, >> >> >>- What is the difference between n_object, n_objectcore & >>n_objecthead? >>- When the objects get expired (or the value of MAIN.n_expired changes) >>will the value of these counters reduce? >> >> >> > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: [NEED HELP] Varnish stops waiting backend after 1 second
Hello, Would you mind showing us the corresponding BeReq? (bereq 3440735) -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish backend dns ttl
Hello there, DNS resolution is done once and for all at VCL load, so you can just reload instead of restart. IIUC, varnish will fail loading the VCL if more than one IP is found though. I would advise you to look ad vmod-named ( https://github.com/Dridi/libvmod-named) and vmod-gwist ( https://github.com/gquintard/libvmod-gwist) that could do what you need. -- Guillaume Quintard On Mon, May 9, 2016 at 4:51 PM, L Cruzero wrote: > In varnish 4.1 will initial DNS resolution during varnishD daemon start up > adhere to a defined backend A record dns ttl, or does one in case of an IP > change to that A record would still need to restart daemon to get the new > IP of the defined backend A record ? > > > > > -LC > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: ESI from remote host not working?
That should work. Possible problems may be that you rewrote req.url before
testing it, or you are setting req.backend_hint (or bereq.backend) later in
the vcl.
can you import std, and log the req.url right before the if-else, and log
right before "set req.backend_hint = esi;"? It would also help if we had
the client part of the log.
--
Guillaume Quintard
On Tue, Apr 26, 2016 at 1:54 PM, Andreas Loewen Internetservices <
kont...@andreas-loewen.de> wrote:
> Dear all,
>
> i am using current varnish varnish 4. I have the following config:
>
> backend esi{
> .host = "x.x.x.x";
> .port = "8081";
> }
>
> ...
>
> sub vcl_recv {
> set req.http.host = "xxx.yyy.de";
> ...
> if (req.url ~ "^/typo3/.*" || req.url ~ "^/secure/.*") {
> set req.backend_hint = typo3be;
> return (synth(403, "Forbidden"));
> } else if (req.url ~ "\.mesi\?" || req.url ~ "\.htm(\?.*)?$" ||
> req.url ~ "^/javax\.faces\..*" || req.url ~ "^/templates/.*" || req.url
> ~"^/f/.*" || req.url ~"^/i/.*") {
> set req.backend_hint = esi;
> set req.hash_always_miss = true;
> } else {
> set req.backend_hint = typo3;
> unset req.http.cookie;
> }
>
> ...
> }
>
> sub vcl_backend_response {
> set beresp.do_esi = true;
> ...
>
> }
>
> The esi backend is on a remote host, but it is reachable from this server.
>
> The funny thing is, that I keep getting 404's for my esi requests for
> /fe_terminliste.mesi?pakid=1 . But they are comming from the default
> backend (local Apache on port 81). varnishlog reads like:
>
> * << BeReq>> 5
> - Begin bereq 4 pass
> - Timestamp Start: 1461670355.552619 0.00 0.00
> - BereqMethodGET
> - BereqURL /fe_terminliste.mesi?pakid=1
> - BereqProtocol HTTP/1.1
> - BereqHeaderCache-Control: max-age=0
> - BereqHeaderAccept:
> text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> - BereqHeaderUpgrade-Insecure-Requests: 1
> - BereqHeaderUser-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36
> - BereqHeaderDNT: 1
> - BereqHeaderAccept-Encoding: gzip, deflate
> - BereqHeaderAccept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
> - BereqHeaderCookie: PHPSESSID=h7hk5n8pa95q4sd2m46ebbju14;
> Typo3InstallTool=tgikum56sujjsv1cu6ra236kb0;
> be_typo_user=5ad26762d12ad4b9fe2d321979fb75e4
> - BereqHeaderHost: xxx:8081
> - BereqHeaderX-Forwarded-For: 80.171.39.71
> - BereqHeaderX-Varnish: 5
> - VCL_call BACKEND_FETCH
> - VCL_return fetch
> - BackendOpen20 default(127.0.0.1,,81) 127.0.0.1 33017
> - Backend20 default default(127.0.0.1,,81)
> - Timestamp Bereq: 1461670355.552877 0.000257 0.000257
> - Timestamp Beresp: 1461670355.553530 0.000911 0.000654
> - BerespProtocol HTTP/1.1
> - BerespStatus 404
> - BerespReason Not Found
> - BerespHeader Date: Tue, 26 Apr 2016 11:32:35 GMT
> - BerespHeader Server: Apache/2.2.15 (CentOS)
> - BerespHeader Content-Length: 310
> - BerespHeader Connection: close
> - BerespHeader Content-Type: text/html; charset=iso-8859-1
> - TTLRFC 120 -1 -1 1461670356 1461670356 1461670355 0 0
> - VCL_call BACKEND_RESPONSE
> - TTLVCL 120 10 0 1461670356
> - VCL_return deliver
> - BerespHeader Content-Encoding: gzip
> - BerespUnsetContent-Length: 310
> - Storagemalloc Transient
> - ObjProtocolHTTP/1.1
> - ObjStatus 404
> - ObjReason Not Found
> - ObjHeader Date: Tue, 26 Apr 2016 11:32:35 GMT
> - ObjHeader Server: Apache/2.2.15 (CentOS)
> - ObjHeader Content-Type: text/html; charset=iso-8859-1
> - ObjHeader Content-Encoding: gzip
> - Fetch_Body 3 length -
> - Gzip G F E 310 268 80 2064 2074
> - Timestamp BerespBody: 1461670355.554030 0.001411 0.000500
> - BackendClose 20 default(127.0.0.1,,81)
> - Length 268
> - BereqAcct 623 0 623 180 310 490
> - End
>
>
> So, does varnish restrict esi to localhost for some reason? Or what is
> wrong?
>
>
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Viewing varnish cache miss urls along with the backend IP
std.log() ? On May 18, 2016 13:51, "Debraj Manna" wrote: > Is there any other way I can achieve the same? > On May 18, 2016 8:10 PM, "Dridi Boukelmoune" wrote: > >> > Is there a way I can also view the backend IPs along with the above >> output? >> >> I don't think there is, varnishtop is a very basic statistics tool. >> >> Dridi >> > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: W3C like format logging
Hi Albert, I'm not familiar to w3c logging, but it looks to me that you are looking for varnishncsa. The format string can be customized and the '-q' parameter will allow you to filter out unwanted request. Also, you can use the std vmod and std.log() to fill a header with HIT|MISS|PASS and retrieve that in varnishncsa. -- Guillaume Quintard On Tue, May 24, 2016 at 12:47 PM, Albert Tollkuçi wrote: > Hello, > I'm new to varnish and I'm setting up for some of the websites I manage. > > One of the difficulties I'm having is to debug it. I know that there's > varnishlog, but it would be really helpful to have something similar to W3C > logging for all request served by varnish. In addition to standard fields, > there will be one extra field to show if the request was served from cache, > from back-end, etc. > > Is there something like this or a way to have this kind of log? > > Thanx, > Albert > > > -- > Web: http://www.tollkuci.com > Follow me on: LinkedIn <http://www.linkedin.com/in/alberttollkuci> Google+ > <https://plus.google.com/+AlbertTollku%C3%A7i/posts> Facebook > <https://www.facebook.com/albert.tollkuci> Twitter > <https://twitter.com/AlbertTollkuci> Career 2.0 > <http://careers.stackoverflow.com/atollkuci> > -- > > Imagination is more important than knowledge > *Albert Einstein* > > The three chief virtues of a programmer are: Laziness, Impatience and > Hubris > *Larry Wall* > > Men are basically smart or dumb and lazy or ambitious. The dumb and > ambitious ones are dangerous and I get rid of them. The dumb and lazy ones > I give mundane duties. The smart ambitious ones I put on my staff. The > smart and lazy ones I make my commanders > *Erwin Rommel* > > The best programmers are not marginally better than merely good ones. > They are an order-of-magnitude better, measured by whatever standard: > conceptual creativity, speed, ingenuity of design, or problem-solving > ability. > *Randall E. Stross* > > Measuring programming progress by lines of code is like measuring aircraft > building progress by weight. > *Bill Gates* > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: VCL debugging
On Tue, May 24, 2016 at 12:51 PM, Albert Tollkuçi wrote: > Hi everyone, > is there a way to log some info from vcl config file? It would help a lot > to log some info on a file to check if the configuration is correct or not. > > > What kind of info do you need? "Correct" is a very broad term :-) ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: W3C like format logging
My bad, you don't need std.log at all (sorry, I was distracted)
It boils down to:
sub vcl_hit {
set req.http.x-state = "HIT";
}
sub vcl_miss {
set req.http.x-state = "MISS";
}
sub vcl_pass {
set req.http.x-state = "PASS";
}
And getting that x-state header in varnishncsa.
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: VCL debugging
On May 24, 2016 17:35, "Albert Tollkuçi" wrote: > > For example, if a cookie is present and I'm testing in vecl_recv some condition, I would like to have the option to log it somewhere, so I know that the condition is met. > Then, here, std.log is an option. Another option is writing a vtc to test this out (to get you started). ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Serving multiple sites from varnish
It should indeed work, IIRC, the hash_data calls are visible in varnishlog, you can start with that. On a side note, please consider trimming down your signature when you post on public mailing-list. As a rule of thumb, your signature shouldn't be longer than your actual message. ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: W3C like format logging
On Wed, May 25, 2016 at 5:35 PM, Albert Tollkuçi wrote: > > Job for varnishncsa.service failed because a configured resource limit was > exceeded. See "systemctl status varnishncsa.service" and "journalctl -xe" > for details. > > Actually, showing us the details would help here, we are in the dark without them. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Serving multiple sites from varnish
On Wed, May 25, 2016 at 5:38 PM, Albert Tollkuçi wrote: > Thank you Guillaume...probably the issue comes from the back-end...will > look into more details. > If you don't find the issue, please post the relevant vcl portion on pastebin, as well as a varnishlog of two requests: one for the desktop, one for the mobile. And post the three links here. > Also, sorry for the signature...fixed it now. > > No problem. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Strange Issue with cache and logged in users
Are you able to isolate the faulty request in varnishlog? The one that
should be passed and isn't?
--
Guillaume Quintard
On Thu, May 26, 2016 at 4:07 PM, Pinakee BIswas wrote:
> Further adding the observations with the following scenario:
>
>1. Visit URL A-not-loggedin
>2. Visit URL B-not-loggedin
>3. User logs in
>4. Visit URL A after logging in - varnish is serving the page -
>A-not-loggedin
>5. Visit URL B after logging in - varnish is serving the page
>B-not-loggedin
>
> This means Varnish is serving the pages just visited (till cache expiry)
> irrespective of the fact that the page is being fetched from backend and
> that there is a pass in vcl_recv.
>
> I am unable to understand the above mentioned behavior - the VCL logic I
> used is simple as mentioned in the mail below.
>
> Regards,
>
> Pinakee
>
> On 26/05/16 6:24 pm, Pinakee BIswas wrote:
>
> Hi,
>
> I am facing a strange problem with Varnish cache. Ours is an ecommerce
> site wherein users can visit with or without logging in. I have written the
> VCL to unset cookies when user not logged in.
>
> The scenario is user logs in and then logs out. When the user logs out,
> Varnish is still delivering logged in pages to the user which is strange as
> the requests for logged in users go all the way to the backend and the
> pages are not supposed to get cached. I have been looking into this since
> quite some time and still not able to figure out the reason.
>
> Would appreciate any help.
>
> Thanks,
>
> Pinakee
>
> PFA the output from varnishlog.
>
> Following is the VCL code:
>
> sub vcl_recv {
> # Happens before we check if we have this in cache already.
> #
> # Typically you clean up the request here, removing cookies you don't
> need,
> # rewriting the request, etc.
> set req.backend_hint = uwsgi;
>
>if (req.method == "GET") {
> if ((req.http.Cookie !~ "sessionid=") &&
> (req.http.Cookie !~ "loggedin_user=") &&
> (req.url !~ "^/esi")) {
> unset req.http.cookie; # strip the cookies - we don't need
> them
> }
> }
> }
>
> sub vcl_backend_response {
> # Happens after we have read the response headers from the backend.
> #
> # Here you clean the response headers, removing silly Set-Cookie
> headers
> # and other mistakes your backend does.
> if (bereq.method == "GET") {
> set beresp.do_esi = true;
> if ((beresp.http.Set-Cookie !~ "jivaana_user=") &&
> (beresp.http.Set-Cookie !~ "sessionid=") &&
> (bereq.http.Cookie !~ "sessionid=") &&
> (bereq.http.Cookie !~ "jivaana_user=") &&
> (bereq.url !~ "^/product/addtobasket")) {
> #unset beresp.http.Set-Cookie;
> set beresp.uncacheable = false;
> return(deliver);
> }
> }
> }
>
>
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Strange Issue with cache and logged in users
On Thu, May 26, 2016 at 4:40 PM, Pinakee BIswas wrote: > Regarding the faulty request, Following are two requests for the same URL > - /heels/ - one when the user was logged in and another when the user had > logged out. The second one (when the user was logged out) is the faulty one > as the user is still seeing the data when he was logged in. > > - BereqMethodGET > - BereqURL /heels/ > - BereqProtocol HTTP/1.0 > - BereqHeaderX-CLIENT-COUNTRY: IN > - BereqHeaderHost: varnish_staging > - BereqHeaderAccept: > text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 > - BereqHeaderUpgrade-Insecure-Requests: 1 > - BereqHeaderUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X > 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 > Safari/537.36 > - BereqHeaderReferer: http://test.jivaana.in/home > - BereqHeaderAccept-Language: en-US,en;q=0.8 > - BereqHeaderX-Forwarded-For: 127.0.0.1 > - BereqHeaderAccept-Encoding: gzip > - BereqProtocol HTTP/1.1 > - BereqHeaderX-Varnish: 32788 > Are you sure your backend isn't caching stuff or something like that? Looking at bereq 32788, varnish is fetch a fresh object, and I see no header that would identify the user. Have you tried without varnish? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: W3C like format logging
On Thu, May 26, 2016 at 4:40 PM, Albert Tollkuçi
wrote:
> Still no solution for this...If i manually run
>
> /usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -F
> "%h|%u|%{%Y-%m-%d}t"
>
> I get the correct log format. However when I put this
> on /etc/systemd/system/varnishncsa.service, such as:
>
>
Then yours seems to be a systemd problem. Do the package-provided script
work? Is SELinux running on your system? If yes, try disabling it just to
test.
--
Guillaume Quintard
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Strange Issue with cache and logged in users
On Thu, May 26, 2016 at 5:29 PM, Pinakee BIswas wrote: > We have been planning to replace nginx caching with Varnish and use nginx > purely as a proxy. For some requests, we plan to bypass Varnish as they > need not be cached. Also, we are serving static content using nginx. > Once this is sorted, I'd rather use varnish as a proxy, but, that's just me :-) > To answer your query, it works fine without varnish. I have tested > removing varnish and then adding it. I also, was surprised that it wasn't > working properly as the logs say otherwise. Will investigate further. > Please do, the logs aren't showing anything weird. Note one thing: varnish will automatically add the X-Varnish header and put the VXID of the request (in case of a MIS/PASS) and the of the bereq (for a HIT), so when you get a wrong object, you can look a the varnishlog (using "-d" to look at past records), possibly filtering it with "-q" and check what was the cached request. One more thing, for the unlogged user, it seems your backend is replying with a set-cookie header. I don't think it's necessary. One more thing is the cookie in the request log isn't showing all the > cookies: > > - ReqHeader Cookie: > _xsrf=2|f25d8caf|ffacf1c86b71827915f94aed8e9aeace|1462920275; > jivaana_country=IN; pagemap=0,0,0,0,0,1,0'; > mp_774636c1ed2371eaf99455f71871069c_mixpanel=%7B%22distinct_id%22%3A%20%221518b8ba6c314b-04bd3d4-6b1b237b-100200-1518b8ba6c428a%22%2C%22%24i > That's expected, varnish will truncate long log records, you can change vsl_reclen to see longer lines in the log. It doesn't change what actually goes on the wire. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish delivering blank page (content-length=0)
Looks like you stumbled on https://github.com/varnishcache/varnish-cache/issues/1954 ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Cache Invalidation
FWIW: a lot of CMS can be configured to warn varnish that content has changed ( http://devdocs.magento.com/guides/v2.0/config-guide/varnish/config-varnish-magento.html ) In this case, the backend will issue HTTP requests to purge content, and of course, your VCL must be ready to treat them. ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Fwd: Varnish Cache
On Fri, Jun 10, 2016 at 11:18 AM, sujith pv wrote: > > So I'm doubtful how we can track the failures in Varnish, do we have any > of the varnishstats indicates us that it has problem and can not serve the > traffic or any other way > That question is actually yours to answer :-) You could use an extra varnish in front of the other and use it as load-balancer, this way you can use the probes to say where to send the request (varnish tier or backend tier). -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnishstat
Hi there, just to be clear: you'd want Varnish to hold the history of counter values in memory for you, right? It's not going to happen, simply because it's not Varnish role to do this, but rather a monitoring solution's. What varnish/varnishstat will give you is a snapshot of the counters, at the time you ask for them. So, if you want to get a value over a period, you'll need to ask for the values at the beginning and end of the period and do the subtraction. However, various tools can help you with that, for example collectd ( https://collectd.org/) works pretty well. I'm sure people on this list will have their own recommandations. Cheers -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Error 503 Backend fetch failed - XID: 208396
Have you checked your probes? If the backend is recognized as sick, indeed Varnish won't even bother sending a request. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Fwd: Varnish Cache
Hum, you can look at the uptime, and see if it resets. You can also look at the dropped/failed sessions and backend failures. Best thing is probably to look at "man varnish-counters" and pick what makes sense. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Log - Logrotate issue
I'm by no mean a logrotate expert, but at the very least, you need to send a SIGUP1 signal to varnishlog to tell it to reopen the file (that should be specified with "-w"). Also, Tom, restarting maybe overkill, and you may miss transactions. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Question about variables
Hi,
What do you mean by "work"? What's the expected result, and what is the
actual result?
--
Guillaume Quintard
On Wed, Jun 29, 2016 at 8:17 AM, Andrei wrote:
> Hello,
>
> I'm currently working on forcing cached results using vsthrottle vs
> dropping requests, but for some reason (I probably did it wrong :) I can't
> get var.get/var.set_duration to work. The vcl_recv snippet is as follows,
> any input is greatly appreciated:
>
> sub vcl_recv {
> if (vsthrottle.is_denied(req.http.Host, 500, 5s) ||
> (var.get("block-" + req.http.Host))) {
> # The vsthrottle rate limit definitely triggers -
> confirmed later with "Attack" header
> unset req.http.Cookie;
> unset req.http.User-Agent;
> unset req.http.Pragma;
> unset req.http.Cache-Control;
> set req.http.Attack = "ByHost: " + req.http.Host;
> set req.ttl = 15m;
> var.set_duration("block-" + req.http.Host,15m);
> return (hash);
> }
> [..]
> }
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish silently rewriting 301 redirect
Hi Josef, Can you reproduce the issue with a minimal VCL and share it? Or give us an anonymized VCL? This looks like a configuration error, Varnish has very little magic happening behind the scenes. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: random 503 + FetchError: straight insufficient bytes
Isn't it because the client stops writing that post request? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Debian Repository
looks like a debian problem. Where did you put the repo definition? Does this file's name ends with ".list"? What happens when you run "aptitude update"? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Question about variables
I haven't checked the code, but it's probably because var.get returns "" when it can't find the variables. So the condition is true, there is a string, even though it's empty. Maybe try with : if (var.get(.) ~ ".") just to make sure it's not empty You can also use sdt.log() to try and print what's inside various variables and debug that using varnishlog. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: random 503 + FetchError: straight insufficient bytes
The client may stop transmitting, for example. Or it could be that the POST request header are wrong/misleading. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Malicious redirection in Varnish?
We actually need the bereq varnishlog to get an idea of what's going on. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Custom TTLs
Hi, the attack header should be available in v_b_r as bereq.http.attack, as req is copied over as bereq and only slightly edited by Varnish (to remove the range-request header, for example). Just a note, if you are attacked, v_b_r is probably a bit late to respond, as the backend as already been bothered. YMWV, of course. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnich Memory in 90% holding continously
Hi, How much RAM do you have on that machine? If the answer is 8GB, it means you allocated all the RAM, leaving nothing for the other applications. What do you mean by "varnish log size"? Do you redirect the output of varnishlog to a file? In that case, it's not suprising, varnishlog is extremely verbose, and you probably want to use varnishncsa instead. Best -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnich Memory in 90% holding continously
"ps aux > foo" to get the full width of the output collected in the foo file. We need the full command lines used too. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Mode: pass all requests
Hello Pablo, I would use vmod-var, and decide if I must pass or not, depending on the value of one of its variables. And I'd have some logic in vcl, protected by acl to allow me to change that value. Does it make sense? ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish threads growing until no response
Sorry, TL;DR, but you have at least one problem: thread_pool_add_delay Value is: 0.000 [seconds] (default) Minimum is: 0.000 Wait at least this long after creating a thread. Some (buggy) systems may need a short (sub-second) delay between creating threads. Set this to a few milliseconds if you see the 'threads_failed' counter grow too much. Setting this too high results in insuffient worker threads. NB: We do not know yet if it is a good idea to change this parameter, or if the default value is even sensible. Caution is advised, and feedback is most welcome. This parameter is now in seconds, that has been know to bite some users. (but thanks to you, I'm only now noticing a typo in the description, thanks!) Can you set thread_pool_add_delay to 0 and let us know how it goes? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Passing Cookies to Drupal Backend
Hi,
By default, Varnish will not cache if the request contains a cookie
headers, or if the response contains a set-cookie header, this is totally
configurable in vcl. Have a look at the builtin.vcl (mine can be found, in
commented form at /usr/share/doc/varnish/builtin.vcl) :
vcl_recv contains:
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
and vcl_backend_response:
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Surrogate-control ~ "no-store" ||
(!beresp.http.Surrogate-Control &&
beresp.http.Cache-Control ~ "no-cache|no-store|private") ||
beresp.http.Vary == "*") {
/*
* Mark as "Hit-For-Pass" for the next 2 minutes
*/
set beresp.ttl = 120s;
set beresp.uncacheable = true;
}
If you don't wish to go through this, you'll have to do a return in your
own vcl, otherwise the builtin.vcl portion of the code gets executed.
To cache part of cookies, check out the vmod-cookies, and in vcl_hash(),
choose the interesting parts for caching.
Does that help?
P.S. : you want to reconsider adding a confidentiality warning that's
longer than your actual message when you post to a public mailing list :-)
--
Guillaume Quintard
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish CPU Usage
can you pastebin your vcl? and maybe a varnishlog? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish CPU Usage
You seem to have a good hit ratio, are you seeing anything on the apache logs that would explain the cpu usage? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish threads growing until no response
Silly question, but have you tried having more threads? And at the same time, thread_pool should be set to 2, unless you have a very valid reason to do otherwise. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish CPU Usage
are you seeing anything on the apache logs that would explain the cpu usage? what's different for apache between with and without varnish? -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Default.vcl + video problem
Please look at varnishlog and check if anything is wrong. You can also trying downloading a video chunk and check if you can play it. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Visitor IP
Have the website look at the X-forwarded-for header? Next varnish version will support the proxy protocol on the backend site, IIUC, so you won't even need the header trick. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Visitor IP
Don't worry about it in vcl, it should already be done automatically by Varnish in the lastest versions. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Cache & Ad Impressions
I'd say no. But maybe you are caching more than you should and screwing up js calls. -- Guillaume Quintard ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Cache & Ad Impressions
Maybe, what is it supposed to do, why is it there and is any of your ad frameworks affected by this? ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish malloc issue
Is that virtual or real memory? Try to cap your Transient storage to 1GB, you may be overloaded with short lived objects. On Sep 1, 2016 9:40 AM, "sujith pv" wrote: > Hi All > > I had already mailed this query long back but this time putting in a > different manner. > > >- We are using Varnish 4.0 in our end. We have a machine with memory >of 16G with 8G being malloc for Varnish. We have a TTL for 2 hrs as well. >- During our peak traffic, when we see the total memory of the machine >reaching 90% and like varnishd process is taking some 89% . >- So I'm just confused even though we had allocated just 8G malloc >which is like 50% of the total memory, how the process is eating up 89% of >memory and the memory is not releasing even after TTL. > > > Any help please... > > Best Regards > Sujith P V > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Sending a ? to the backend
Did should work, have a look at varnishlog (Req AND corresponding Bereq),
it will give you a better view of what is going on. If in doubt, pastebin
it, and send us the link.
And obviously, make sure you have nothing else in you vcl that could
interfer with that if.
--
Guillaume Quintard
On Thu, Sep 1, 2016 at 7:58 PM, Ian Macdonald wrote:
> Hi All,
>
> I am having trouble rewriting a url I need to send a question mark to
> the backend and it just does not seem to work I have tried escaping
> the ?, long strings, regsub() and all seem to fail. here is a snippet
> if anyone can point me in the right direction
>
> vcl_recv {
> ...
>
> if (req.url ~ "^/myurl$") {
>
> set req.url = {"/search?q=term"};
> return (pass);
>
>
>
>
> ...
>
> }
>
> quick httpie check gets seems to show anything from the ? goes missing
>
> The requested URL /search was not found on this
> server. That’s all we know.
>
> varnish is 4.1.3 on centos 6.8
>
> Cheers
>
> Ian
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish malloc issue
Hi Viktor, Have a look here https://www.varnish-cache.org/docs/trunk/users-guide/storage-backends.html#transient-storage Sujith, please try to use a packaged version from your distribution. -- Guillaume Quintard On Fri, Sep 2, 2016 at 1:18 AM, Viktor Villafuerte < viktor.villafue...@optusnet.com.au> wrote: > Hi Guillaume, > > On Thu 01 Sep 2016 09:53:22, Guillaume Quintard wrote: > > Is that virtual or real memory? > > Try to cap your Transient storage to 1GB, you may be overloaded with > short > > lived objects. > > Could you elaborate on this bit more? I've got very similar problem > here. Eg: Total memory 384g > > VIRT: 487g > RES: 311g > > free says: 46g free > > but SWAP is 91.3% used :( > > > How can this be? > > > thanks > > v > > > > > > On Sep 1, 2016 9:40 AM, "sujith pv" wrote: > > > > > Hi All > > > > > > I had already mailed this query long back but this time putting in a > > > different manner. > > > > > > > > >- We are using Varnish 4.0 in our end. We have a machine with memory > > >of 16G with 8G being malloc for Varnish. We have a TTL for 2 hrs as > well. > > >- During our peak traffic, when we see the total memory of the > machine > > >reaching 90% and like varnishd process is taking some 89% . > > >- So I'm just confused even though we had allocated just 8G malloc > > >which is like 50% of the total memory, how the process is eating up > 89% of > > >memory and the memory is not releasing even after TTL. > > > > > > > > > Any help please... > > > > > > Best Regards > > > Sujith P V > > > > > > ___ > > > varnish-misc mailing list > > > varnish-misc@varnish-cache.org > > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > > > ___ > > varnish-misc mailing list > > varnish-misc@varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > -- > Regards > > Viktor Villafuerte > Optus Internet Engineering > t: +61 2 80825265 > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Cache & Ad Impressions
ss );
>>}
>>
>> ## Do not cache search results, comment these 3 lines if you do want
>> to cache them
>>
>> if (req.url ~ "/\?s\=") {
>> return ( pass );
>> }
>>
>> # CLEAN UP THE ENCODING HEADER.
>> # SET TO GZIP, DEFLATE, OR REMOVE ENTIRELY. WITH VARY ACCEPT-ENCODING
>> # VARNISH WILL CREATE SEPARATE CACHES FOR EACH
>> # DO NOT ACCEPT-ENCODING IMAGES, ZIPPED FILES, AUDIO, ETC.
>> # ##
>> if (req.http.Accept-Encoding) {
>> if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
>> # No point in compressing these
>> unset req.http.Accept-Encoding;
>> } elsif (req.http.Accept-Encoding ~ "gzip") {
>> set req.http.Accept-Encoding = "gzip";
>> } elsif (req.http.Accept-Encoding ~ "deflate") {
>> set req.http.Accept-Encoding = "deflate";
>> } else {
>> # unknown algorithm
>> unset req.http.Accept-Encoding;
>> }
>> }
>>
>> # PIPE ALL NON-STANDARD REQUESTS
>> # ##
>> if (req.method != "GET" &&
>> req.method != "HEAD" &&
>> req.method != "PUT" &&
>> req.method != "POST" &&
>> req.method != "TRACE" &&
>> req.method != "OPTIONS" &&
>> req.method != "DELETE") {
>> return (pipe);
>> }
>>
>> # ONLY CACHE GET AND HEAD REQUESTS
>> # ##
>> if (req.method != "GET" && req.method != "HEAD") {
>> return (pass);
>> }
>>
>> # OPTIONAL: DO NOT CACHE LOGGED IN USERS (THIS OCCURS IN FETCH TOO,
>> EITHER
>> # COMMENT OR UNCOMMENT BOTH
>> # ##
>> if ( req.http.cookie ~ "wordpress_logged_in" ) {
>> return( pass );
>> }
>>
>> # IF THE REQUEST IS NOT FOR A PREVIEW, WP-ADMIN OR WP-LOGIN
>> # THEN UNSET THE COOKIES
>> # ##
>> if (!(req.url ~ "wp-(login|admin)")
>> && !(req.url ~ "&preview=true" )
>> ){
>> unset req.http.cookie;
>> }
>>
>> # IF BASIC AUTH IS ON THEN DO NOT CACHE
>> # ##
>> if (req.http.Authorization || req.http.Cookie) {
>> return (pass);
>> }
>>
>> # IF YOU GET HERE THEN THIS REQUEST SHOULD BE CACHED
>> # ##
>> return (hash);
>> # This is for phpmyadmin
>> if (req.http.Host == "ki1.org") {
>> return (pass);
>> }
>>
>> if (req.http.Host == "mysql.ki1.org") {
>> return (pass);
>> }
>>
>> }
>>
>> # HIT FUNCTION
>> # ##
>> sub vcl_hit {
>> # IF THIS IS A PURGE REQUEST THEN DO THE PURGE
>> # ##
>> if (req.method == "PURGE") {
>> #
>> # This is now handled in vcl_recv.
>> #
>> # purge;
>> return (synth(200, "Purged."));
>> }
>> return (deliver);
>> }
>>
>> # MISS FUNCTION
>> # ##
>> sub vcl_miss {
>> if (req.method == "PURGE") {
>> #
>> # This is now handled in vcl_recv.
>> #
>> # purge;
>> return (synth(200, "Purged."));
>> }
>> return (fetch);
>> }
>>
>> # FETCH FUNCTION
>> # ##
>> sub vcl_backend_response {
>> # I SET THE VARY TO ACCEPT-ENCODING, THIS OVERRIDES W3TC
>> # TENDANCY TO SET VARY USER-AGENT. YOU MAY OR MAY NOT WANT
>> # TO DO THIS
>> # ##
>> set beresp.http.Vary = "Accept-Encoding";
>>
>> # IF NOT WP-ADMIN THEN UNSET COOKIES AND SET THE AMOUNT OF
>> # TIME THIS PAGE WILL STAY CACHED (TTL)
>> # ##
>>if (!(bereq.url ~ "wp-(login|admin)|forum") && !bereq.http.cookie ~
>> "wordpress_logged_in" && !bereq.http.host == "**.com/forum" ) {
>> unset beresp.http.set-cookie;
>> set beresp.ttl = 52w;
>> #set beresp.grace =1w;
>> }
>>
>>
>>
>> if (beresp.ttl <= 0s ||
>> beresp.http.Set-Cookie ||
>> beresp.http.Vary == "*") {
>> set beresp.ttl = 120 s;
>> # set beresp.ttl = 120s;
>> set beresp.uncacheable = true;
>> return (deliver);
>> }
>>
>> return (deliver);
>> }
>>
>> # DELIVER FUNCTION
>> # ##
>> sub vcl_deliver {
>> # IF THIS PAGE IS ALREADY CACHED THEN RETURN A 'HIT' TEXT
>> # IN THE HEADER (GREAT FOR DEBUGGING)
>> # ##
>> if (obj.hits > 0) {
>> set resp.http.X-Cache = "HIT";
>> # IF THIS IS A MISS RETURN THAT IN THE HEADER
>> # ##
>> } else {
>> set resp.http.X-Cache = "MISS";
>> }
>> }
>>
>>
>> 2016-08-16 19:42 GMT+03:00 Guillaume Quintard <
>> guilla...@varnish-software.com>:
>>
>>> Maybe, what is it supposed to do, why is it there and is any of your ad
>>> frameworks affected by this?
>>>
>>
>>
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Caching for some cookies?
Hi,
Look at the varnishlog, there should be a trace of that unset, once
confirmed, find it in your vcl.
Know that if you don't return from a subroutine, the built-in vcl is
executed ("locate builtin.vcl" to find a commented version of it).
On Sep 1, 2016 12:59, "Albert Tollkuçi" wrote:
> Hello,
> I've started using Varnish a few months ago and my vcl files has started
> to get complicated. I'm serving 5 different sites with the same varnish
> instance (using different backends) and this part is working.
>
> Now I have some cases that for some sites I need to keep some cookies and
> cache different versions based on them. Basic scenario is to show the user
> a message only the first time he sees the page. Without varnish if the
> cookie is not there I set it and show the message. Next time the cookie is
> there and no message is shown.
>
> With varnish my logic is to include the cookie in hash and cache two
> different versions. The challenge is to make sure cookies are set correctly
> and here I'm kind of stack. I've added some debug messages in my vcl and
> for some reason I'm not seeing the cookies I set in my server in
> vcl_backend_response. Inside the routine, I have the following code in the
> beginning:
>
># AT: DEBUG
>if(bereq.http.x-at-debug == "1") {
> std.log("AT DEBUG - bereq URL: " + bereq.url + ". Cookies: '" +
> bereq.http.Cookie + "'");
> std.log("AT DEBUG - beresp set-cookie: '" +
> beresp.http.set-cookie + "'");
>}
>
> I set x-at-debug header in vcl_recv based on my IP, so I can track only my
> requests. In the server side I see the cookie "__IKUB_AL_Myikub_PopUp"
> being set, but in vcl_backend_response is not.
>
> Is there anything in between that can strip the cookie?
>
> Thank you,
> Albert
>
>
>
> --
> Web: http://www.tollkuci.com
> Follow me on: LinkedIn Google+
> Facebook
> Twitter
> Career 2.0
>
>
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish malloc issue
https://www.varnish-cache.org/docs/4.1/reference/varnishd.html#shortlived On Sep 6, 2016 08:07, "sujith pv" wrote: > Thanks Guillaume. I will try the transient settings first and verify the > same. Also could you please elaborate on the short lived objects. > > Best Regards > Sujith P V > > On Fri, Sep 2, 2016 at 11:24 AM, Guillaume Quintard < > guilla...@varnish-software.com> wrote: > >> Hi Viktor, >> >> Have a look here https://www.varnish-cache.org/docs/trunk/users-guide/ >> storage-backends.html#transient-storage >> >> Sujith, please try to use a packaged version from your distribution. >> >> -- >> Guillaume Quintard >> >> On Fri, Sep 2, 2016 at 1:18 AM, Viktor Villafuerte < >> viktor.villafue...@optusnet.com.au> wrote: >> >>> Hi Guillaume, >>> >>> On Thu 01 Sep 2016 09:53:22, Guillaume Quintard wrote: >>> > Is that virtual or real memory? >>> > Try to cap your Transient storage to 1GB, you may be overloaded with >>> short >>> > lived objects. >>> >>> Could you elaborate on this bit more? I've got very similar problem >>> here. Eg: Total memory 384g >>> >>> VIRT: 487g >>> RES: 311g >>> >>> free says: 46g free >>> >>> but SWAP is 91.3% used :( >>> >>> >>> How can this be? >>> >>> >>> thanks >>> >>> v >>> >>> >>> > >>> > On Sep 1, 2016 9:40 AM, "sujith pv" wrote: >>> > >>> > > Hi All >>> > > >>> > > I had already mailed this query long back but this time putting in a >>> > > different manner. >>> > > >>> > > >>> > >- We are using Varnish 4.0 in our end. We have a machine with >>> memory >>> > >of 16G with 8G being malloc for Varnish. We have a TTL for 2 hrs >>> as well. >>> > >- During our peak traffic, when we see the total memory of the >>> machine >>> > >reaching 90% and like varnishd process is taking some 89% . >>> > >- So I'm just confused even though we had allocated just 8G malloc >>> > >which is like 50% of the total memory, how the process is eating >>> up 89% of >>> > >memory and the memory is not releasing even after TTL. >>> > > >>> > > >>> > > Any help please... >>> > > >>> > > Best Regards >>> > > Sujith P V >>> > > >>> > > ___ >>> > > varnish-misc mailing list >>> > > varnish-misc@varnish-cache.org >>> > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> > > >>> >>> > ___ >>> > varnish-misc mailing list >>> > varnish-misc@varnish-cache.org >>> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >>> >>> -- >>> Regards >>> >>> Viktor Villafuerte >>> Optus Internet Engineering >>> t: +61 2 80825265 >>> >> >> > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish malloc issue
Sorry, I didn't get your question. -- Guillaume Quintard On Tue, Sep 6, 2016 at 8:31 AM, sujith pv wrote: > Thanks Guillaume. May be Im asking a very basic question , but still how > this short lived objects are created and I'm not giving any settings for > the same as well. > > On Tue, Sep 6, 2016 at 11:54 AM, Guillaume Quintard < > guilla...@varnish-software.com> wrote: > >> https://www.varnish-cache.org/docs/4.1/reference/varnishd.html#shortlived >> >> On Sep 6, 2016 08:07, "sujith pv" wrote: >> >>> Thanks Guillaume. I will try the transient settings first and verify >>> the same. Also could you please elaborate on the short lived objects. >>> >>> Best Regards >>> Sujith P V >>> >>> On Fri, Sep 2, 2016 at 11:24 AM, Guillaume Quintard < >>> guilla...@varnish-software.com> wrote: >>> >>>> Hi Viktor, >>>> >>>> Have a look here https://www.varnish-cache >>>> .org/docs/trunk/users-guide/storage-backends.html#transient-storage >>>> >>>> Sujith, please try to use a packaged version from your distribution. >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> On Fri, Sep 2, 2016 at 1:18 AM, Viktor Villafuerte < >>>> viktor.villafue...@optusnet.com.au> wrote: >>>> >>>>> Hi Guillaume, >>>>> >>>>> On Thu 01 Sep 2016 09:53:22, Guillaume Quintard wrote: >>>>> > Is that virtual or real memory? >>>>> > Try to cap your Transient storage to 1GB, you may be overloaded with >>>>> short >>>>> > lived objects. >>>>> >>>>> Could you elaborate on this bit more? I've got very similar problem >>>>> here. Eg: Total memory 384g >>>>> >>>>> VIRT: 487g >>>>> RES: 311g >>>>> >>>>> free says: 46g free >>>>> >>>>> but SWAP is 91.3% used :( >>>>> >>>>> >>>>> How can this be? >>>>> >>>>> >>>>> thanks >>>>> >>>>> v >>>>> >>>>> >>>>> > >>>>> > On Sep 1, 2016 9:40 AM, "sujith pv" wrote: >>>>> > >>>>> > > Hi All >>>>> > > >>>>> > > I had already mailed this query long back but this time putting in >>>>> a >>>>> > > different manner. >>>>> > > >>>>> > > >>>>> > >- We are using Varnish 4.0 in our end. We have a machine with >>>>> memory >>>>> > >of 16G with 8G being malloc for Varnish. We have a TTL for 2 >>>>> hrs as well. >>>>> > >- During our peak traffic, when we see the total memory of the >>>>> machine >>>>> > >reaching 90% and like varnishd process is taking some 89% . >>>>> > >- So I'm just confused even though we had allocated just 8G >>>>> malloc >>>>> > >which is like 50% of the total memory, how the process is >>>>> eating up 89% of >>>>> > >memory and the memory is not releasing even after TTL. >>>>> > > >>>>> > > >>>>> > > Any help please... >>>>> > > >>>>> > > Best Regards >>>>> > > Sujith P V >>>>> > > >>>>> > > ___ >>>>> > > varnish-misc mailing list >>>>> > > varnish-misc@varnish-cache.org >>>>> > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>>> > > >>>>> >>>>> > ___ >>>>> > varnish-misc mailing list >>>>> > varnish-misc@varnish-cache.org >>>>> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>>> >>>>> >>>>> -- >>>>> Regards >>>>> >>>>> Viktor Villafuerte >>>>> Optus Internet Engineering >>>>> t: +61 2 80825265 >>>>> >>>> >>>> >>> > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish statistics
Have a look at collectd, I'd say. I also so a few people using zabbix -- Guillaume Quintard On Thu, Aug 18, 2016 at 12:22 PM, Miguel González < miguel_3_gonza...@yahoo.es> wrote: > Hi! > > I have a WHM/Cpanel server behind Varnish. I would like to know which > are the websites that use up the most of the Varnish cache (I´m thinking > of even creating different storages for the most used websites). > > I took a look into Munin and Pandora, but maybe you guys know simpler > or better tools? I don´t mind if they are GUIs or command line tools. > > Regards, > > Miguel > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish threads growing until no response
Glad to hear that, thanks a lot for coming back to us! -- Guillaume Quintard On Mon, Sep 12, 2016 at 12:10 PM, Michael Dosser wrote: > Hi, > > Varnish is now 45 days up and running. Since that is by far the longest > uptime in this configuration I think we can close this issue. > > Thanks again for your help! > > Best regards, > Michael Dosser > > > Am 10.08.2016 um 16:23 schrieb Michael Dosser : > > > > Hi, > > > > just an update - since we made those changes our Varnish instance is > still up and running. I did some other changes: > > > > - Removed „set beresp.ttl = 0s;“ in vcl_backend_response and added > default_ttl=0 in Varnish startup > > - Removed the whole inline C-Code - replaced with Varnish own s-maxage > handling > > > > I’ll keep you informed if this solved our problems … > > > > Michael Dosser > > > >> Am 29.07.2016 um 14:44 schrieb Michael Dosser : > >> > >> Hi, > >> > >>> Am 29.07.2016 um 14:20 schrieb Guillaume Quintard < > guilla...@varnish-software.com>: > >>> > >>> Silly question, but have you tried having more threads? And at the > same time, thread_pool should be set to 2, unless you have a very valid > reason to do otherwise. > >> > >> thanks for your help. I’ll try these new settings! I didn’t think about > changing the startup logic and always thought our problem is coming from > our quite complex VCL. > >> > >> Michael Dosser > >> > >> PS: I changed the startup to: > >> > >> varnishd -f /etc/varnish/fuf.vcl \ > >> -h critbit \ > >> -a 127.0.0.1:6081 \ > >> -T 127.0.0.1:6082 \ > >> -t 120 \ > >> -S /etc/varnish/secret \ > >> -s malloc,12G \ > >> -p thread_pool_min=500 \ > >> -p thread_pool_max=3000 \ > >> -p thread_pools=2 \ > >> -p thread_pool_add_delay=0 \ > >> -p vcc_allow_inline_c=on \ > >> -p feature=+esi_disable_xml_check \ > >> -p feature=+esi_ignore_other_elements \ > >> -p feature=+esi_ignore_https" > >> > >> -- > >> strg.at gmbh michael.dos...@strg.at > >> gumpendorferstrasse 132, top 9, 1060 wien > >> tel +43 (1) 526 56 29 mobile +43 699 1 164 > >> > >> > >> ___ > >> varnish-misc mailing list > >> varnish-misc@varnish-cache.org > >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > > -- > > strg.at gmbh michael.dos...@strg.at > >gumpendorferstrasse 132, top 9, 1060 wien > >tel +43 (1) 526 56 29 mobile +43 699 1 164 > > > > > > ___ > > varnish-misc mailing list > > varnish-misc@varnish-cache.org > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > -- > strg.at gmbh michael.dos...@strg.at > gumpendorferstrasse 132, top 9, 1060 wien > tel +43 (1) 526 56 29 mobile +43 699 1 164 > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish malloc issue
What about the other g_space/g_bytes? On Sep 14, 2016 18:33, "sujith pv" wrote: > It was showing as 0 for Transient_g_bytes and 1Gb for g_space as I > mentioned in my mail. > > On Wed, Sep 14, 2016 at 7:45 PM, Guillaume Quintard < > guilla...@varnish-software.com> wrote: > >> please keep using the mailing list, some other people may have ideas. >> >> what's the output of "varnishstat -1 | grep g_bytes" ? >> >> -- >> Guillaume Quintard >> >> On Wed, Sep 14, 2016 at 3:50 PM, sujith pv wrote: >> >>> Hi Guillaume >>> >>> I have tried capping Transient and I could see the respective values >>> under SMA.transient.g_bytes/g_space. But still on load I didnt see this >>> g_bytes being used and was showing 0 only. >>> >>> Best Regards >>> Sujith P V >>> >>> On Tue, Sep 6, 2016 at 7:39 PM, Guillaume Quintard < >>> guilla...@varnish-software.com> wrote: >>> >>>> Maybe, not sure. Unless you have a **good** reason to do so, use the >>>> packages. >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> On Tue, Sep 6, 2016 at 3:19 PM, sujith pv wrote: >>>> >>>>> Thanks Guillaume. Actually in my end Varnish was compiled here and >>>>> installed (I was not part of that initially :-)). Will this have any >>>>> impact >>>>> compared to as you have mentioned like try from a packaged version. >>>>> >>>>> On Tue, Sep 6, 2016 at 6:46 PM, Guillaume Quintard < >>>>> guilla...@varnish-software.com> wrote: >>>>> >>>>>> It's just a question of storage, shortlive objects got into transient >>>>>> storage instead of the one(s) you defined in the command line. But it's >>>>>> still a cache storage, based on malloc, that is by default unlimited in >>>>>> size. >>>>>> >>>>>> The rationale is that for object with such short ttl+grace+keep, a >>>>>> simple malloc is good enough. >>>>>> >>>>>> -- >>>>>> Guillaume Quintard >>>>>> >>>>>> On Tue, Sep 6, 2016 at 3:04 PM, sujith pv >>>>>> wrote: >>>>>> >>>>>>> Actually I'm confused around the short lived objects. Is these >>>>>>> objects are something like cached objects or who is creating these >>>>>>> objects. >>>>>>> >>>>>>> On Tue, Sep 6, 2016 at 1:57 PM, Guillaume Quintard < >>>>>>> guilla...@varnish-software.com> wrote: >>>>>>> >>>>>>>> Sorry, I didn't get your question. >>>>>>>> >>>>>>>> -- >>>>>>>> Guillaume Quintard >>>>>>>> >>>>>>>> On Tue, Sep 6, 2016 at 8:31 AM, sujith pv >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Guillaume. May be Im asking a very basic question , but >>>>>>>>> still how this short lived objects are created and I'm not giving any >>>>>>>>> settings for the same as well. >>>>>>>>> >>>>>>>>> On Tue, Sep 6, 2016 at 11:54 AM, Guillaume Quintard < >>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>> >>>>>>>>>> https://www.varnish-cache.org/docs/4.1/reference/varnishd.ht >>>>>>>>>> ml#shortlived >>>>>>>>>> >>>>>>>>>> On Sep 6, 2016 08:07, "sujith pv" wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Guillaume. I will try the transient settings first and >>>>>>>>>>> verify the same. Also could you please elaborate on the short lived >>>>>>>>>>> objects. >>>>>>>>>>> >>>>>>>>>>> Best Regards >>>>>>>>>>> Sujith P V >>>>>>>>>>> >>>>>>>>>>> On Fri, Sep 2, 2016 at 11:24 AM, Guillaume Quintard < >>>>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>>>> >>&g
Re: Varnish malloc issue
Have you tried the packages from your distribution? On Sep 14, 2016 20:09, "sujith pv" wrote: > After the full load of requests..g_space became minimal and g_bytes became > like 8 Gb..I had assigned malloc as 8 gb > > On 14-Sep-2016 22:42, "Guillaume Quintard" > wrote: > >> What about the other g_space/g_bytes? >> >> On Sep 14, 2016 18:33, "sujith pv" wrote: >> >>> It was showing as 0 for Transient_g_bytes and 1Gb for g_space as I >>> mentioned in my mail. >>> >>> On Wed, Sep 14, 2016 at 7:45 PM, Guillaume Quintard < >>> guilla...@varnish-software.com> wrote: >>> >>>> please keep using the mailing list, some other people may have ideas. >>>> >>>> what's the output of "varnishstat -1 | grep g_bytes" ? >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> On Wed, Sep 14, 2016 at 3:50 PM, sujith pv wrote: >>>> >>>>> Hi Guillaume >>>>> >>>>> I have tried capping Transient and I could see the respective values >>>>> under SMA.transient.g_bytes/g_space. But still on load I didnt see this >>>>> g_bytes being used and was showing 0 only. >>>>> >>>>> Best Regards >>>>> Sujith P V >>>>> >>>>> On Tue, Sep 6, 2016 at 7:39 PM, Guillaume Quintard < >>>>> guilla...@varnish-software.com> wrote: >>>>> >>>>>> Maybe, not sure. Unless you have a **good** reason to do so, use the >>>>>> packages. >>>>>> >>>>>> -- >>>>>> Guillaume Quintard >>>>>> >>>>>> On Tue, Sep 6, 2016 at 3:19 PM, sujith pv >>>>>> wrote: >>>>>> >>>>>>> Thanks Guillaume. Actually in my end Varnish was compiled here and >>>>>>> installed (I was not part of that initially :-)). Will this have any >>>>>>> impact >>>>>>> compared to as you have mentioned like try from a packaged version. >>>>>>> >>>>>>> On Tue, Sep 6, 2016 at 6:46 PM, Guillaume Quintard < >>>>>>> guilla...@varnish-software.com> wrote: >>>>>>> >>>>>>>> It's just a question of storage, shortlive objects got into >>>>>>>> transient storage instead of the one(s) you defined in the command >>>>>>>> line. >>>>>>>> But it's still a cache storage, based on malloc, that is by default >>>>>>>> unlimited in size. >>>>>>>> >>>>>>>> The rationale is that for object with such short ttl+grace+keep, a >>>>>>>> simple malloc is good enough. >>>>>>>> >>>>>>>> -- >>>>>>>> Guillaume Quintard >>>>>>>> >>>>>>>> On Tue, Sep 6, 2016 at 3:04 PM, sujith pv >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Actually I'm confused around the short lived objects. Is these >>>>>>>>> objects are something like cached objects or who is creating these >>>>>>>>> objects. >>>>>>>>> >>>>>>>>> On Tue, Sep 6, 2016 at 1:57 PM, Guillaume Quintard < >>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>> >>>>>>>>>> Sorry, I didn't get your question. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Guillaume Quintard >>>>>>>>>> >>>>>>>>>> On Tue, Sep 6, 2016 at 8:31 AM, sujith pv >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Guillaume. May be Im asking a very basic question , but >>>>>>>>>>> still how this short lived objects are created and I'm not giving >>>>>>>>>>> any >>>>>>>>>>> settings for the same as well. >>>>>>>>>>> >>>>>>>>>>> On Tue, Sep 6, 2016 at 11:54 AM, Guillaume Quintard < >>>>>>>>>>> guilla...@varnish-software.com> wrote: >
Re: Varnish malloc issue
That can't, hurt. On Sep 15, 2016 03:31, "sujith pv" wrote: > No..I'm planning to try that next. So during that time also so do we need > to cap transient storage ?? > > On 15-Sep-2016 01:18, "Guillaume Quintard" > wrote: > >> Have you tried the packages from your distribution? >> >> On Sep 14, 2016 20:09, "sujith pv" wrote: >> >>> After the full load of requests..g_space became minimal and g_bytes >>> became like 8 Gb..I had assigned malloc as 8 gb >>> >>> On 14-Sep-2016 22:42, "Guillaume Quintard" < >>> guilla...@varnish-software.com> wrote: >>> >>>> What about the other g_space/g_bytes? >>>> >>>> On Sep 14, 2016 18:33, "sujith pv" wrote: >>>> >>>>> It was showing as 0 for Transient_g_bytes and 1Gb for g_space as I >>>>> mentioned in my mail. >>>>> >>>>> On Wed, Sep 14, 2016 at 7:45 PM, Guillaume Quintard < >>>>> guilla...@varnish-software.com> wrote: >>>>> >>>>>> please keep using the mailing list, some other people may have ideas. >>>>>> >>>>>> what's the output of "varnishstat -1 | grep g_bytes" ? >>>>>> >>>>>> -- >>>>>> Guillaume Quintard >>>>>> >>>>>> On Wed, Sep 14, 2016 at 3:50 PM, sujith pv >>>>>> wrote: >>>>>> >>>>>>> Hi Guillaume >>>>>>> >>>>>>> I have tried capping Transient and I could see the respective values >>>>>>> under SMA.transient.g_bytes/g_space. But still on load I didnt see this >>>>>>> g_bytes being used and was showing 0 only. >>>>>>> >>>>>>> Best Regards >>>>>>> Sujith P V >>>>>>> >>>>>>> On Tue, Sep 6, 2016 at 7:39 PM, Guillaume Quintard < >>>>>>> guilla...@varnish-software.com> wrote: >>>>>>> >>>>>>>> Maybe, not sure. Unless you have a **good** reason to do so, use >>>>>>>> the packages. >>>>>>>> >>>>>>>> -- >>>>>>>> Guillaume Quintard >>>>>>>> >>>>>>>> On Tue, Sep 6, 2016 at 3:19 PM, sujith pv >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks Guillaume. Actually in my end Varnish was compiled here and >>>>>>>>> installed (I was not part of that initially :-)). Will this have any >>>>>>>>> impact >>>>>>>>> compared to as you have mentioned like try from a packaged version. >>>>>>>>> >>>>>>>>> On Tue, Sep 6, 2016 at 6:46 PM, Guillaume Quintard < >>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>> >>>>>>>>>> It's just a question of storage, shortlive objects got into >>>>>>>>>> transient storage instead of the one(s) you defined in the command >>>>>>>>>> line. >>>>>>>>>> But it's still a cache storage, based on malloc, that is by default >>>>>>>>>> unlimited in size. >>>>>>>>>> >>>>>>>>>> The rationale is that for object with such short ttl+grace+keep, >>>>>>>>>> a simple malloc is good enough. >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Guillaume Quintard >>>>>>>>>> >>>>>>>>>> On Tue, Sep 6, 2016 at 3:04 PM, sujith pv >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Actually I'm confused around the short lived objects. Is these >>>>>>>>>>> objects are something like cached objects or who is creating these >>>>>>>>>>> objects. >>>>>>>>>>> >>>>>>>>>>> On Tue, Sep 6, 2016 at 1:57 PM, Guillaume Quintard < >>>>>>>>>>> guilla...@varnish-software.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Sorr
Re: Recommendation for varnishlog in production
1. use logrotate, or something like that 2. varnishlog is probably overkill for what you need, turn your gaze towards varnishncsa 3. if you really want to use varnishlog, you should use '-q' to filter only the "important" transactions (slow fetches/deliveries, errors, etc.) -- Guillaume Quintard On Mon, Sep 19, 2016 at 2:37 PM, Albert Tollkuçi wrote: > I've setup varnish as a proxy for a few websites and it's running > smoothly. However I did notice that the log file is huge (daily size of > around 100 GB). I guess that so much logging will hurt the performance and > I'm looking at different options. > > I tried to remove -w filename option but this was required. Probably > another option is to disable the service. However before making the changes > I would like to ask if there's any recommendation of how to setup > varnishlog in production server. > > Thank you, > Albert > > -- > Web: http://www.tollkuci.com > Follow me on: LinkedIn <http://www.linkedin.com/in/alberttollkuci> Google+ > <https://plus.google.com/+AlbertTollku%C3%A7i/posts> Facebook > <https://www.facebook.com/albert.tollkuci> Twitter > <https://twitter.com/AlbertTollkuci> Career 2.0 > <http://careers.stackoverflow.com/atollkuci> > -- > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: varnish + gatling http first read error: EOF
Look at varnishlog, and find the error in the log. And possibly look at a few BeReq before that. I've seen apache wrongly replying 304 with body. As Varnish expect no body, it bytes are read as the next request's headers, and as you can guess, that doesn't work well. You may be affected my a variant of that. On Sep 21, 2016 11:14, "Tobias Honacker" wrote: > Hi all, > > we are facing some issues trying to stresstest our oAuth App using Varnish > 4.0.3 and Gatling > > Infra: > > LB -> Apache (SSL Offloading) + Varnish -> Appserver (Apache + Tomcat) > > Varnishlog: > http://pastebin.com/tCWgi01H > > VCL: > http://pastebin.com/cEdzAwxs > > tcpdump on our app server with 1 req/s (varnishlog 200): > http://pastebin.com/XFVsANiN > > tcpdump on our app server with >= 2 req/s (varnishlog 503er): > http://pastebin.com/f2EKADpx > > If varnish throws HTTP 503 the connection: close header is missing > > Running tests using pipe works fine, because pipe sets connection: close > Running tests without varnish works fine, too. > > The Apaches using KeepAlive: > > KeepAlive On > MaxKeepAliveRequests 100 > KeepAliveTimeout 10 > > > Any suggestions? > > Thanks in advance and best regards > Tobias > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Is varnish agent is an open source what are the dependency so required
Are you using the ubuntu packages or compiling from source? -- Guillaume Quintard On Wed, Sep 21, 2016 at 12:31 PM, akshay accha wrote: > I am not able to install varnish-agent what dependencies are required and > what is the procedure to install varnish-agent such that i already have > varnish 4.1 installed and have ubuntu as os. I have the vagent2-master file > in /etc/vagent2-master how to install varnish agent from here ? > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Is varnish agent is an open source what are the dependency so required
I don't know what you are talking about, what was the URL you downloaded it from? -- Guillaume Quintard On Wed, Sep 21, 2016 at 4:44 PM, akshay accha wrote: > I have downloaded the master file and moved it to the destination folder > as mentioned ,how do I proceed ? > > > On Wednesday, 21 September 2016, Guillaume Quintard < > guilla...@varnish-software.com> wrote: > >> Are you using the ubuntu packages or compiling from source? >> >> -- >> Guillaume Quintard >> >> On Wed, Sep 21, 2016 at 12:31 PM, akshay accha >> wrote: >> >>> I am not able to install varnish-agent what dependencies are required >>> and what is the procedure to install varnish-agent such that i already have >>> varnish 4.1 installed and have ubuntu as os. I have the vagent2-master file >>> in /etc/vagent2-master how to install varnish agent from here ? >>> ___ >>> varnish-misc mailing list >>> varnish-misc@varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> >> ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Is varnish agent is an open source what are the dependency so required
Try installing the varnish-agent package, it'll be much easier. Please, keep varnish-misc address in the CC when you reply. -- Guillaume Quintard On Wed, Sep 21, 2016 at 5:08 PM, akshay accha wrote: > Url for varnish agent I downloaded the master file from > https://github.com/varnish/vagent2 . > > Varnish 4.1 I installed using Ubuntu package and I tried installing > varnishagent but it could not develope dependency tree so I downloaded > master file and tried installing it. > > Could you please tell me how to install varnish agent given that I already > have varnish 4.1 installed using Ubuntu package > > On Wednesday, 21 September 2016, Guillaume Quintard < > guilla...@varnish-software.com> wrote: > >> I don't know what you are talking about, what was the URL you downloaded >> it from? >> >> -- >> Guillaume Quintard >> >> On Wed, Sep 21, 2016 at 4:44 PM, akshay accha >> wrote: >> >>> I have downloaded the master file and moved it to the destination folder >>> as mentioned ,how do I proceed ? >>> >>> >>> On Wednesday, 21 September 2016, Guillaume Quintard < >>> guilla...@varnish-software.com> wrote: >>> >>>> Are you using the ubuntu packages or compiling from source? >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> On Wed, Sep 21, 2016 at 12:31 PM, akshay accha >>> > wrote: >>>> >>>>> I am not able to install varnish-agent what dependencies are required >>>>> and what is the procedure to install varnish-agent such that i already >>>>> have >>>>> varnish 4.1 installed and have ubuntu as os. I have the vagent2-master >>>>> file >>>>> in /etc/vagent2-master how to install varnish agent from here ? >>>>> ___ >>>>> varnish-misc mailing list >>>>> varnish-misc@varnish-cache.org >>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>>> >>>> >>>> >> ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: I have installed varnish-agent but it has some issues.
I do see it, look again? -- Guillaume Quintard On Thu, Sep 22, 2016 at 6:51 AM, akshay accha wrote: > varnish-agent -H option is not working and when i type man varnish-agent > the -H option is not there what might be the solution? > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Does varnish 3.09 support varnish-agent -H and varnishlog -g option?
Sir, you got to stop spamming this mailing list. We also have an IRC channel, consider using it. And no, varnishlog -g is only supported stating v4.0. the varnish-agent shouldn't be affected by the varnish version. -- Guillaume Quintard On Thu, Sep 22, 2016 at 10:53 AM, akshay accha wrote: > > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: bitrate and memory allocated
Focus on g_space (free space in the storage) and g_bytes (used bytes in the storage). -- Guillaume Quintard On Tue, Sep 6, 2016 at 12:08 PM, Miguel González wrote: > On 09/05/16 10:44 PM, Dridi Boukelmoune wrote: > >> Any URL? I have googled for a few hours but I didn´t get a clear > >> understanding from the official manuals. > > > > https://www.varnish-cache.org/docs/4.1/reference/varnish- > counters.html#per-malloc-storage-counters-sma > > > I had read the 4.0 docs but I find them quite short and not very > explanatory. What are the counters? The current use of memory? The > amount allocated since Varnish was restarted (historical)? At least for > me, I would like an example to better understand what those counters mean. > > Thanks > > Miguel > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Problem Upgrade to Varnish 4.1 - Multiple Backends Not Working
What does "stop working" mean? Does the vcl fail to compile? do you receive
503s? does it open a portal to an ancient dimension filled with unspoken
nightmares? Help us help you.
--
Guillaume Quintard
On Fri, Sep 23, 2016 at 1:52 PM, Drew, AJ wrote:
> Hello,
>
> We are trying to upgrade to Varnish 4.1 from Varnish 3.0.5.
>
> Most things are working fine, but a big problem we are running into is
> that we have multiple backends. We switch backends based on URLs. This
> was working fine in Varnish 3, but when we switched over to Varnish 4, it
> stopped working.
>
> I have tried many variations of the URLs, and even simplified the
> directors so that they only contain one backend (in my tests) so that I
> could switch from trying the backend directly to trying the director.
>
> Using either of the options does not seem to matter.
>
> Has anyone else run into this problem?
>
> Here are the sections of my VCL that contain the backends / director
> definitions and the if statements to switch. I have also left in the
> different options (lines) I have used to come up with different
> combinations for testing.
>
> probe B_probe {
> .request =
> "GET /zz/status.php HTTP/1.1"
> "HOST: xx.example.com"
> "Connection: close";
> .timeout = 0.3 s;
> .window= 8;
> .threshold = 7;
> }
>
> backend B_backend1 {
> .host = "162.131.196.220";
> .port = "8080";
> .probe = B_probe;
> }
>
> backend syscheck {
> #.host = "localhost";
> .host = "127.0.0.1";
> .port = "8081";
> }
>
> backend searchBackend {
> .host = "rp.www.example.com";
> .port = "80";
> }
>
> backend providerBackend {
> .host = "172.23.82.28";
> .port = "80";
> }
>
> sub vcl_init {
> new examplehosts = directors.round_robin();
> examplehosts.add_backend(B_backend1);
>
> new examplesyscheck = directors.round_robin();
> examplesyscheck.add_backend(syscheck);
> }
>
>
>
>
> sub vcl_recv {
>
> # set req.backend_hint = examplehosts.backend();
> set req.backend_hint = B_backend1;
>
> if (req.url ~ "^/syscheck/.*$") {
> # if (req.url ~ "^/syscheck/*") {
> # if (req.url ~ "^/syscheck/") {
> set req.backend_hint = syscheck;
> # set req.backend_hint = examplesyscheck.backend();
> return(pass);
> }
>
> # Add simple health check for upstream load balancers
> if (req.url ~ "^/health-check") {
> return( synth(200, "UP"));
> }
> ...
>
>
> Thanks!
>
> A J Drew
>
>
>
> -Message Disclaimer-
>
> This e-mail message is intended only for the use of the individual or
> entity to which it is addressed, and may contain information that is
> privileged, confidential and exempt from disclosure under applicable law.
> If you are not the intended recipient, any dissemination, distribution or
> copying of this communication is strictly prohibited. If you have received
> this communication in error, please notify us immediately by reply email to
> conn...@principal.com and delete or destroy all copies of the original
> message and attachments thereto. Email sent to or from the Principal
> Financial Group or any of its member companies may be retained as required
> by law or regulation.
>
> Nothing in this message is intended to constitute an Electronic signature
> for purposes of the Uniform Electronic Transactions Act (UETA) or the
> Electronic Signatures in Global and National Commerce Act ("E-Sign") unless
> a specific statement to the contrary is included in this message.
>
> If you no longer wish to receive any further solicitation from the
> Principal Financial Group you may unsubscribe at
> https://www.principal.com/do-not-contact-form any time.
>
> If you are a Canadian resident and no longer wish to receive commercial
> electronic messages you may unsubscribe at https://www.principal.com/do-
> not-email-request-canadian-residents any time.
>
>
> ___
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
___
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Problem Upgrade to Varnish 4.1 - Multiple Backends Not Working
They look good, but make sure your regex are doing the way you want, and to debug, put some "std.log()" statement to make sure the request is triggering the right ifs. Also, if that doesn't help, please provide use with the Request section (as you did) as well as the BeReq section associated (in this case "2 Link c bereq 3 pass" -> BeReq 3). Having the whole VCL (on pastebin) would also help. -- Guillaume Quintard On Fri, Sep 23, 2016 at 3:20 PM, Drew, AJ wrote: > Sorry for not being for specific. > > > > The initial backend being set is the one that is always used no matter > what the URL. > > > > So, when I come in with a URL of /syscheck/blah.htm, I would expect the > alternative backend to be used. It does not get select, and instead the > original one is the one used, and therefore I end up with either a HTTP 404 > or a HTTP 503, depending on the mood of Drupal the destination of the > initial backend. > > > > Thanks! > > > > A J Drew > > > > > > Here is some output from varnishlog of a call: > > > > 2 Begin c req 1 rxreq > > 2 Timestamp c Start: 1474629889.600227 0.00 0.00 > > 2 Timestamp c Req: 1474629889.600227 0.00 0.00 > > 2 ReqStart c 127.0.0.1 34608 > > 2 ReqMethod c GET > > 2 ReqURL c /syscheck/wwwul2370.htm > > 2 ReqProtocolc HTTP/1.1 > > 2 ReqHeader c Host: xx.example.com > > 2 ReqHeader c Accept: text/html, application/xhtml+xml, */* > > 2 ReqHeader c Accept-Language: en-US > > 2 ReqHeader c User-Agent: Mozilla/5.0 (Windows NT 6.1; > WOW64; Trident/7.0; rv:11.0) like Gecko > > 2 ReqHeader c Accept-Encoding: gzip, deflate > > 2 ReqHeader c Cookie: ObSSOCookie=loggedout; TLTSID= > A5D86DBA4DA97DFA943BD59DA9EFF5B8; dtCookie=3A7E953F591088FA2F6DB23E157B3A > 54|Q1NTX19FQV9fYXV0aG9yaXphdGlvbi1oYW5kbGVyLWVhcnwxfENTU19fRUFf > X2FjY291bnQtYWdncmVnYXRpb24tZWFyfDE; dtLatC=36971; dtPC=-; > RANDOM_ID=6529725af06 > > 2 ReqHeader c Via: 1.1 xx.example.com > > 2 ReqHeader c X-Forwarded-For: 172.22.159.116 > > 2 ReqHeader c X-Forwarded-Host: xx.example.com > > 2 ReqHeader c X-Forwarded-Server: xx.example.com > > 2 ReqHeader c Connection: Keep-Alive > > 2 ReqUnset c X-Forwarded-For: 172.22.159.116 > > 2 ReqHeader c X-Forwarded-For: 172.22.159.116, 127.0.0.1 > > 2 VCL_call c RECV > > 2 VCL_return c pass > > 2 VCL_call c HASH > > 2 VCL_return c lookup > > 2 VCL_call c PASS > > 2 VCL_return c fetch > > 2 Link c bereq 3 pass > > 2 Timestamp c Fetch: 1474629891.152683 1.552456 1.552456 > > 2 RespProtocol c HTTP/1.1 > > 2 RespStatus c 404 > > 2 RespReason c Not Found > > 2 RespHeader c Date: Fri, 23 Sep 2016 11:24:49 GMT > > 2 RespHeader c Server: Apache > > 2 RespHeader c X-Drupal-Cache: MISS > > 2 RespHeader c Expires: Sun, 19 Nov 1978 05:00:00 GMT > > 2 RespHeader c Cache-Control: public, max-age=3600 > > 2 RespHeader c Content-Language: en > > 2 RespHeader c X-Generator: Drupal 7 (http://drupal.org) > > 2 RespHeader c Etag: "1474629889-0" > > 2 RespHeader c Last-Modified: Fri, 23 Sep 2016 11:24:49 GMT > > 2 RespHeader c Vary: Cookie,Accept-Encoding > > 2 RespHeader c Strict-Transport-Security: max-age=31536000; > includeSubDomains; preload > > 2 RespHeader c X-XSS-Protection: 1; > > 2 RespHeader c Content-Type: text/html; charset=utf-8 > > 2 RespHeader c X-Varnish: 2 > > 2 RespHeader c Age: 0 > > 2 RespHeader c Via: 1.1 varnish-v4 > > 2 VCL_call c DELIVER > > 2 RespHeader c X-Cache: MISS > > 2 RespUnset c Via: 1.1 varnish-v4 > > 2 RespUnset c X-Generator: Drupal 7 (http://drupal.org) > > 2 RespUnset c Server: Apache > > 2 VCL_return c deliver > > 2 Timestamp c Process: 1474629891.152737 1.552510 0.54 > > 2 RespHeader c Content-Length: 7685 > > 2 Debug c "RES_MODE 2" > > 2 RespHeader c Conne
Re: bitrate and memory allocated
Your varnishstat doesn't seem to indicate that (s0.g_bytes + s0.g_space = 100M). Can you give us the output of "ps aux | grep varnish" ? -- Guillaume Quintard On Sun, Sep 25, 2016 at 10:14 PM, Miguel González < miguel_3_gonza...@yahoo.es> wrote: > Still I don´t understand the numbers. I have set the Varnish storage in > memory of 6 Gb: > > SMA.s0.c_req 1040458 3.53 Allocator > requests > SMA.s0.c_fail 715580 2.43 Allocator > failures > SMA.s0.c_bytes 8027903959 27242.60 Bytes allocated > SMA.s0.c_freed 7923307928 26887.65 Bytes freed > SMA.s0.g_alloc5217 . Allocations > outstanding > SMA.s0.g_bytes 104596031 . Bytes > outstanding > SMA.s0.g_space 261569 . Bytes available > SMA.Transient.c_req 236994 0.80 Allocator > requests > SMA.Transient.c_fail 0 0.00 Allocator > failures > SMA.Transient.c_bytes 2047296885 6947.48 Bytes > allocated > SMA.Transient.c_freed 2047294773 6947.47 Bytes freed > SMA.Transient.g_alloc6 . Allocations > outstanding > SMA.Transient.g_bytes 2112 . Bytes > outstanding > SMA.Transient.g_space0 . Bytes available > > > > On 09/23/16 11:16 AM, Guillaume Quintard wrote: > > Focus on g_space (free space in the storage) and g_bytes (used bytes in > > the storage). > > > > -- > > Guillaume Quintard > > > > On Tue, Sep 6, 2016 at 12:08 PM, Miguel González > > mailto:miguel_3_gonza...@yahoo.es>> wrote: > > > > On 09/05/16 10:44 PM, Dridi Boukelmoune wrote: > > >> Any URL? I have googled for a few hours but I didn´t get a clear > > >> understanding from the official manuals. > > > > > > https://www.varnish-cache.org/docs/4.1/reference/varnish- > counters.html#per-malloc-storage-counters-sma > > <https://www.varnish-cache.org/docs/4.1/reference/ > varnish-counters.html#per-malloc-storage-counters-sma> > > > > > > I had read the 4.0 docs but I find them quite short and not very > > explanatory. What are the counters? The current use of memory? The > > amount allocated since Varnish was restarted (historical)? At least > for > > me, I would like an example to better understand what those counters > > mean. > > > > Thanks > > > > Miguel > > > > ___ > > varnish-misc mailing list > > varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish- > cache.org> > > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc> > > > > > > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Different host headers - same cache?
On Oct 16, 2016 01:15, "Thomas Lecomte" wrote: > > Hello, > > On Sat, Oct 15, 2016 at 11:03 AM, Frederik Ramm wrote: > > Question 1: > > You will need to rewrite bereq.http.host in the vcl_backend_fetch > routine depending on the backend to which you are sending the request. I think it's easier than that: just remove the host header and let varnish add the right one, as this (old) ticket seems to indicate: http://varnish-cache.org/trac/ticket/230 . I was assuming the same thing as Frederik: that host_header was a hard override. I think a docfix is in order (ie. I'll do it) ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Variable for Decoded req.http.Authorization
Hi Kelvin, It's actually pretty simple to do: use a regex to get the second word, then use vmod-var to decode the token. On Oct 19, 2016 12:15, "Kelvin Loke" wrote: > Good day everyday, I have a question about req.http.Authorization. > As req.http.Authorization is at the format of "Basic Uk1CVTUweWMjIyMg==", > do we have any variable for the base64 decoded value from > req.http.Authorization? > > I knew that Varnishncsa has %u for this purpose, it would be great if > there is a default variable that I can use in Varnish VCL. > > Thanks! > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Variable for Decoded req.http.Authorization
Arg, my bad, that was indeed vmod-digest (somehow, my tablet thinks vmod-var is an acceptable autocorrection for vmod-digest) On Oct 20, 2016 06:05, "Kelvin Loke" wrote: > Hi Kelvin, >> >> It's actually pretty simple to do: use a regex to get the second word, >> then use vmod-var to decode the token. >> > I did check vmod-var, I couldn't find any usage/manual about the base64 > decode function. Do you mean the libvmod-digest? > > Cheers. > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Error 503 Backend fetch failed
Have you changed http_resp_size too? -- Guillaume Quintard On Mon, Oct 17, 2016 at 9:23 AM, Guilleret Florian < guilleret.flor...@gmail.com> wrote: > Hi, > > I'm getting weird 503 error on varnish. > > On page with ESI ( not on every page of my website ) i got Error 503 > Backend fetch failed. > > On varnish log i can saw FetchError http read error: overflow : > > > * << BeReq >> 9 > - Begin bereq 8 fetch > - Timestamp Start: 1476370852.736703 0.00 0.00 > - BereqMethod GET > - BereqURL /pageXXX.html > - BereqProtocol HTTP/1.1 > - BereqHeader Host: www.website.com > - BereqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:47.0) > Gecko/20100101 Firefox/47.0 > - BereqHeader Accept: text/html,application/xhtml+ > xml,application/xml;q=0.9,*/*;q=0.8 > - BereqHeader Accept-Language: fr-FR > - BereqHeader Cookie: mage-translation-storage=%7B%7D; > mage-translation-file-version=%7B%7D; mage-cache-storage=%7B%7D; > mage-cache-storage-section-invalidation=%7B%7D; mage-cache-sessid=true; > X-Magento-Vary=f970ba1c8edf4251be121a7535f2ce003b791953; > form_key=lV5GbPY6uP > - BereqHeader X-Forwarded-For: 80.74.72.196 > - BereqHeader Accept-Encoding: gzip > - BereqHeader X-Varnish: 9 > - VCL_call BACKEND_FETCH > - VCL_return fetch > - BackendOpen 53 default(127.0.0.1,,81) 127.0.0.1 59720 > - Backend 53 default default(127.0.0.1,,81) > - Timestamp Bereq: 1476370852.736959 0.000256 0.000256 > - FetchError http read error: overflow > - BackendClose 53 default(127.0.0.1,,81) > - Timestamp Beresp: 1476370865.707061 12.970358 12.970102 > - Timestamp Error: 1476370865.707072 12.970368 0.11 > - BerespProtocol HTTP/1.1 > - BerespStatus 503 > - BerespReason Service Unavailable > - BerespReason Backend fetch failed > - BerespHeader Date: Thu, 13 Oct 2016 15:01:05 GMT > - BerespHeader Server: Varnish > - VCL_call BACKEND_ERROR > - BerespHeader Content-Type: text/html; charset=utf-8 > - BerespHeader Retry-After: 5 > - VCL_return deliver > - Storage malloc Transient > - ObjProtocol HTTP/1.1 > - ObjStatus 503 > - ObjReason Backend fetch failed > - ObjHeader Date: Thu, 13 Oct 2016 15:01:05 GMT > - ObjHeader Server: Varnish > - ObjHeader Content-Type: text/html; charset=utf-8 > - ObjHeader Retry-After: 5 > - Length 278 > - BereqAcct 611 0 611 7767 0 7767 > - End > > > My website is a magento 2. So I suppose this is my issue : > > http://devdocs.magento.com/guides/v2.0/config-guide/ > varnish/tshoot-varnish-503.html > > But even with this option http_resp_hdr_len it doesn't work... > > Can anyone know how to debug this ? > > ___ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
