Re: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching
Thanks, so, to make things clean you are going to need to use a couple of vmods, which means being able to compile them first: - https://github.com/otto-de/libvmod-uuid as Geoff offered - https://github.com/Dridi/libvmod-querystring that will allow easy manipulation of the querystring unfortunately, the install-vmod tool that is bundled into the Varnish docker image isn't able to cleanly compile/install them. I'll have a look this week-end if I can, or at least I'll open a ticket on https://github.com/varnish/docker-varnish But, if you are able to install those two, then your life is easy: - once you receive a request, you can start by creating a unique ID, which'll be the the vcl equivalent of `uuidgen | sed -E 's/(\w+)-(\w+)-(\w+)-(\w+).*/\1\2\3\4/'` (without having testing it, probably `regsub(uuid.uuid_v4(), "s/(\w+)-(\w+)-(\w+)-(\w+).*", "\1\2\3\4/"`) - then just add/replace the parameter in the querystring with vmod_querystring and...that's about it? Problem is getting the vmods to compile/install which I can help with this week-end. There's black magic that you can do using regex to manipulate querystring, but it's a terrible idea. -- Guillaume Quintard On Wed, May 31, 2023 at 6:48 PM Uday Kumar wrote: > > Does it need to be unique? can't we just get away with >> ""? >> > > Our Requirements: > 1. New Parameter should be *appended *to already existing parameters in > Query String. (should not replace entire query string) > 2. Parameter Value *Must be Unique for each request* (ideally unique > randomness is preferred) > 3. Allowed Characters are Alphanumeric which are *URL safe* [can be > lowercase, uppercase in case of alphabets] > 4. Characters can be repeated in parameter value EX: Gn4lT*Y*gBgpPaRi6hw6 > *Y*S (here, Y is repeated) But as mentioned above the value must be > unique as a whole. > > Ex: Parameter value for 1st request can be "Gn4lT*Y*gBgpPaRi6hw6*Y*S", > 2nd request can be > "G34lTYgBgpPaRi6hyaaF" and so on > > > Thanks & Regards > Uday Kumar > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching
> Does it need to be unique? can't we just get away with > ""? > Our Requirements: 1. New Parameter should be *appended *to already existing parameters in Query String. (should not replace entire query string) 2. Parameter Value *Must be Unique for each request* (ideally unique randomness is preferred) 3. Allowed Characters are Alphanumeric which are *URL safe* [can be lowercase, uppercase in case of alphabets] 4. Characters can be repeated in parameter value EX: Gn4lT*Y*gBgpPaRi6hw6*Y*S (here, Y is repeated) But as mentioned above the value must be unique as a whole. Ex: Parameter value for 1st request can be "Gn4lT*Y*gBgpPaRi6hw6*Y*S", 2nd request can be "G34lTYgBgpPaRi6hyaaF" and so on Thanks & Regards Uday Kumar ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching
On 5/31/23 20:06, Uday Kumar wrote: We would like to configure varnish to create unique parameter such that its value should be of 20 characters (alphanumeric characters that are URL safe). I have used VMOD uuid to set unique values in each request: https://github.com/otto-de/libvmod-uuid The standard UUID string is 36 characters long. If you really have to have exactly 20 characters, you could take the last 3 blocks of hex characters, removing the hyphens. It would take a bit of regex slinging. For example from: 6ba7b810-9dad-11d1-80b4-00c04fd430c8 extract: 11d180b400c04fd430c8 Then you'd have 80 bits represented in 20 hex characters, instead of the 128 bits represented by the standard string. The 80 bits are all obtained from the random number generator (the urandom device on Linux), so I suppose that would be enough to ensure uniqueness. Best, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstraße 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de OpenPGP_signature Description: OpenPGP digital signature ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching
Does it need to be unique? can't we just get away with ""? the crude VCL code would look like: set req.url = regsub(req.url, "\?.*","?yourparam="); i.e. getting rid of the whole query string and just putting yours in place. -- Guillaume Quintard On Wed, May 31, 2023 at 11:07 AM Uday Kumar wrote: > Hello, > > We would like to configure varnish to create unique parameter such that > its value should be of 20 characters (alphanumeric characters that are URL > safe). > > On Wed, May 31, 2023, 13:34 Guillaume Quintard < > guillaume.quint...@gmail.com> wrote: > >> > Could you please also suggest how to configure Varnish so that Varnish >> can add Unique Parameter by itself?? >> >> We'd need more context, is there any kind of check that tomcat does on >> this parameter, does it need to have a specific length, or match a regex? >> If we know that, we can have Varnish check the user request to make sure >> it's valid, and potentially generate its own parameter. >> >> But it all depends on what Tomcat expects from that parameter. >> >> -- >> Guillaume Quintard >> >> >> On Tue, May 30, 2023 at 11:18 PM Uday Kumar >> wrote: >> >>> Hello Guillaume, >>> >>> Thank you so much for your help, will try modifying vcl_hash as >>> suggested! >>> >>> Last note: it would probably be better if the tomcat server didn't need that unique parameter, or at the very least, if Varnish could just add it itself rather than relying on client information as you're caching something public using something that was user-specific, so there's potential for snafus here. >>> >>> Could you please also suggest how to configure Varnish so that Varnish >>> can add Unique Parameter by itself?? >>> >> ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching
Hello, We would like to configure varnish to create unique parameter such that its value should be of 20 characters (alphanumeric characters that are URL safe). On Wed, May 31, 2023, 13:34 Guillaume Quintard wrote: > > Could you please also suggest how to configure Varnish so that Varnish > can add Unique Parameter by itself?? > > We'd need more context, is there any kind of check that tomcat does on > this parameter, does it need to have a specific length, or match a regex? > If we know that, we can have Varnish check the user request to make sure > it's valid, and potentially generate its own parameter. > > But it all depends on what Tomcat expects from that parameter. > > -- > Guillaume Quintard > > > On Tue, May 30, 2023 at 11:18 PM Uday Kumar > wrote: > >> Hello Guillaume, >> >> Thank you so much for your help, will try modifying vcl_hash as suggested! >> >> >>> Last note: it would probably be better if the tomcat server didn't need >>> that unique parameter, or at the very least, if Varnish could just add >>> it itself rather than relying on client information as you're caching >>> something public using something that was user-specific, so there's >>> potential for snafus here. >>> >> >> Could you please also suggest how to configure Varnish so that Varnish >> can add Unique Parameter by itself?? >> > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching
> Could you please also suggest how to configure Varnish so that Varnish can add Unique Parameter by itself?? We'd need more context, is there any kind of check that tomcat does on this parameter, does it need to have a specific length, or match a regex? If we know that, we can have Varnish check the user request to make sure it's valid, and potentially generate its own parameter. But it all depends on what Tomcat expects from that parameter. -- Guillaume Quintard On Tue, May 30, 2023 at 11:18 PM Uday Kumar wrote: > Hello Guillaume, > > Thank you so much for your help, will try modifying vcl_hash as suggested! > > >> Last note: it would probably be better if the tomcat server didn't need >> that unique parameter, or at the very least, if Varnish could just add >> it itself rather than relying on client information as you're caching >> something public using something that was user-specific, so there's >> potential for snafus here. >> > > Could you please also suggest how to configure Varnish so that Varnish > can add Unique Parameter by itself?? > ___ varnish-misc mailing list varnish-misc@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
