Hi, I'm pretty sure there's a confusion with the sequence of actions here. Normalization happen *before* you look into the cache, so that way before you fetch anything from the backend. By the time you cache the data (vcl_backend_response), the hash key has already been set (vcl_hash), it's way too late to normalize the request.
As to normalization, it's usually done in vcl_recv, and it can range from just setting the host header to a static string to using std.tolower() and removing the host port. for the sake of the example: sub vcl_vcl { set req.http.host = "myvideoservice.com"; } For shard example, look at the VCTs, for example: https://github.com/varnishcache/varnish-cache/blob/6.6/bin/varnishtest/tests/d00029.vtc#L66 import directors; sub vcl_init { new shard_dir = directors.shard(); shard_dir.add_backend(be1); shard_dir.add_backend(be2); shard_dir.add_backend(be3; new p = directors.shard_param(); vd.associate(p.use()); vd.reconfigure(replicas=25);} sub vcl_backend_fetch { p.set(by=KEY, key=bereq.url); set bereq.backend_hint = shard_dir.backend(resolve=LAZY);} For udo: import crypto; import udo; sub vcl_init { new udo_dir = udo.director(); udo_dir.set_type(random); udo_dir.add_backend(be1); udo_dir.add_backend(be2); udo_dir.add_backend(be3); udo_dir.set_type(hash);} sub vcl_backend_fetch { set bereq.backend_hint = udo_dir.backend(); udo_dir.set_hash(crypto.hash(sha256, bereq.url));} These have been written without testing, so don't put them straight into production. -- Guillaume Quintard On Thu, Aug 5, 2021 at 3:33 AM Hamidreza Hosseini <hrhosse...@hotmail.com> wrote: > Hi, > 1. > > Is there any way to normalize host headers and other things to say to > varnish not to cache the same content for different backend? > I want to use round robin director but after fetching the content I want > to normalize the header and cache the content, > I would appreciate if you give me an example about this and how I can do > it. > > 2. > I couldn't find any good example for directors-shard and > xshard-key-string, I would appreciate if you could give example about this > too. > > Many Thanks > ------------------------------ > *From:* varnish-misc <varnish-misc-bounces+hrhosseini= > hotmail....@varnish-cache.org> on behalf of Hamidreza Hosseini < > hrhosse...@hotmail.com> > *Sent:* Sunday, August 1, 2021 4:17 AM > *To:* varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org> > *Subject:* Best practice for caching scenario with different backend > servers but same content > > Hi, > I want to use varnish in my scenario as cache service, I have about 10 > http servers that serve Hls fragments as the backend servers and about 5 > varnish servers for caching purpose, the problem comes in when I use > round-robin director for backend servers in varnish, > if a varnish for specific file requests to one backend server and for the > same file but to another backend server it would cache that file again > because of different Host headers ! so my solution is using fallback > director instead of round-robin as follow: > > ``` > In varnish-1: > new hls_cluster = directors.fallback(); > hls_cluster.add_backend(b1()); > hls_cluster.add_backend(b2()); > hls_cluster.add_backend(b3()); > hls_cluster.add_backend(b4()); > hls_cluster.add_backend(b5()); > hls_cluster.add_backend(b6()); > hls_cluster.add_backend(b7()); > hls_cluster.add_backend(b8()); > hls_cluster.add_backend(b9()); > hls_cluster.add_backend(b10()); > > > > In varnish-2: > new hls_cluster = directors.fallback(); > hls_cluster.add_backend(b10()); > hls_cluster.add_backend(b1()); > hls_cluster.add_backend(b2()); > hls_cluster.add_backend(b3()); > hls_cluster.add_backend(b4()); > hls_cluster.add_backend(b5()); > hls_cluster.add_backend(b6()); > hls_cluster.add_backend(b7()); > hls_cluster.add_backend(b8()); > hls_cluster.add_backend(b9()); > > > In varnish-3: > new hls_cluster = directors.fallback(); > hls_cluster.add_backend(b9()); > hls_cluster.add_backend(b1()); > hls_cluster.add_backend(b2()); > hls_cluster.add_backend(b3()); > hls_cluster.add_backend(b4()); > hls_cluster.add_backend(b5()); > hls_cluster.add_backend(b6()); > hls_cluster.add_backend(b7()); > hls_cluster.add_backend(b8()); > hls_cluster.add_backend(b10()); > > ``` > But I think this is not the best solution, because there is no load > balancing despite, I used different backend for the first argument of > fallback directive, > What is varnish recommendation for this scenario? > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc@varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >
_______________________________________________ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc