Re: [squid-users] Question about the url rewrite before proxy out

[Bill Yuan]

Did not resolve yet. anyone can help to provide an example plase



On 23 September 2016 at 07:57, Bill Yuan  wrote:

> Thanks for replying
>
> I see, it is a url rewrite  and i am trying to find an sample to have a
> test
>
>
> the content are not required to be updated if it is a image
>
>
> On Thursday, September 22, 2016, Alex Rousskov <
> rouss...@measurement-factory.com> wrote:
>
>> On 09/22/2016 04:20 AM, squid-us...@filter.luko.org wrote:
>>
>> > Then not only the request needs to be rewritten, but probably the
>> > page content too. [...] If that is the case,
>> > then Squid doesn't seem like the right tool for the job.
>>
>> Why not? If rewriting is needed, an ICAP or eCAP service can rewrite the
>> response body before Squid serves it.
>>
>> However, in practice, this kind of page rewriting does not work very
>> well (regardless of which software is doing the rewrite) because many
>> page URLs are formed dynamically on the client side (by Javascript code).
>>
>> Alex.
>>
>> ___
>> squid-users mailing list
>> squid-users@lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
>>
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Question about the url rewrite before proxy out

[Bill Yuan]

Thanks for replying

I see, it is a url rewrite  and i am trying to find an sample to have a test


the content are not required to be updated if it is a image


On Thursday, September 22, 2016, Alex Rousskov <
rouss...@measurement-factory.com> wrote:

> On 09/22/2016 04:20 AM, squid-us...@filter.luko.org  wrote:
>
> > Then not only the request needs to be rewritten, but probably the
> > page content too. [...] If that is the case,
> > then Squid doesn't seem like the right tool for the job.
>
> Why not? If rewriting is needed, an ICAP or eCAP service can rewrite the
> response body before Squid serves it.
>
> However, in practice, this kind of page rewriting does not work very
> well (regardless of which software is doing the rewrite) because many
> page URLs are formed dynamically on the client side (by Javascript code).
>
> Alex.
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org 
> http://lists.squid-cache.org/listinfo/squid-users
>
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Question about the url rewrite before proxy out

[Alex Rousskov]

On 09/22/2016 04:20 AM, squid-us...@filter.luko.org wrote:

> Then not only the request needs to be rewritten, but probably the 
> page content too. [...] If that is the case,
> then Squid doesn't seem like the right tool for the job.

Why not? If rewriting is needed, an ICAP or eCAP service can rewrite the
response body before Squid serves it.

However, in practice, this kind of page rewriting does not work very
well (regardless of which software is doing the rewrite) because many
page URLs are formed dynamically on the client side (by Javascript code).

Alex.

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Question about the url rewrite before proxy out

[squid-users]

> 
> 
> > If you input http://www.yahoo.com/page.html, this will be transformed
> > to http://192.168.1.1/www.google.com/page.html.
> 
> I got the impression that the OP wanted the rewrite to work the other way
> around.

My apologies, that does seem to be the case.

> Squid sees http://192.168.1.1/www.google.com and  re-writes it to
> http://www.google.com
> 
> > The helper just needs to print that out prepended by "OK rewrite-
> url=xxx".
> > More info at
> > http://www.squid-cache.org/Doc/config/url_rewrite_program/
> >
> > Of course, you will need something listening on 192.168.1.1 (Apache,
> > nginx,
> > whatever) that can deal with those rewritten requests.
> 
> I got the impression that the OP wanted Squid to be listening on this
> address, doing the rewrites, and then fetching from standard origin
> servers.

Then not only the request needs to be rewritten, but probably the page content 
too.  Eg, assets in the page will all be pointing at 
http://www.yahoo.com/image.png and also need transforming to 
http://192.168.1.1/www.yahoo.com/image.png.

If that is the case, then Squid doesn't seem like the right tool for the job.  
I think CGIproxy can do this (https://www.jmarshall.com/tools/cgiproxy/) or 
perhaps Apache's mod_proxy 
(https://httpd.apache.org/docs/current/mod/mod_proxy.html) would work.

Luke


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Question about the url rewrite before proxy out

[Antony Stone]

On Thursday 22 Sep 2016 at 06:04, squid-us...@filter.luko.org wrote:

> > i am looking for a proxy which can "bounce" the request, which is not a
> > classic proxy.
> > 
> > I want it works in this way.
> > 
> > e.g. a proxy is running a 192.168.1.1
> > and when i want to open http://www.yahoo.com, i just need call
> > http://192.168.1.1/www.yahoo.com the proxy can pickup the the host
> > "http://www.yahoo.com; from the URI, and retrieve the info for me​, so
> > it need to get the new $host from $location, and remove the $host from
> > the $location before proxy pass it. it is doable via squid?
> 
> Yes it is doable (but unusual).  First you need to tell Squid which requests
> should be rewritten, then send them to a rewrite program to be transformed. 
> Identify the domains like this:



> If you input http://www.yahoo.com/page.html, this will be transformed to
> http://192.168.1.1/www.google.com/page.html.

I got the impression that the OP wanted the rewrite to work the other way 
around.

Squid sees http://192.168.1.1/www.google.com and  re-writes it to 
http://www.google.com

> The helper just needs to print that out prepended by "OK rewrite-url=xxx". 
> More info at http://www.squid-cache.org/Doc/config/url_rewrite_program/
> 
> Of course, you will need something listening on 192.168.1.1 (Apache, nginx,
> whatever) that can deal with those rewritten requests.

I got the impression that the OP wanted Squid to be listening on this address, 
doing the rewrites, and then fetching from standard origin servers.

> That is an unusual way of getting requests to 192.168.1.1 though, because
> you are effectively putting the hostname component into the URL then sending
> it to a web service and expecting it to deal with that.

Yes, that's what the OP wants Squid to handle, I think.


Antony.

-- 
"640 kilobytes (of RAM) should be enough for anybody."

 - Bill Gates

   Please reply to the list;
 please *don't* CC me.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] Question about the url rewrite before proxy out

[squid-users]

> i am looking for a proxy which can "bounce" the request, which is not a 
> classic proxy.
>
> I want it works in this way.
> 
> e.g. a proxy is running a 192.168.1.1 
> and when i want to open http://www.yahoo.com, i just need call 
> http://192.168.1.1/www.yahoo.com
> the proxy can pickup the the host "http://www.yahoo.com; from the URI, and 
> retrieve the info for me​, 
> so it need to get the new $host from $location, and remove the $host from the 
> $location before proxy pass it.
> it is doable via squid?

Yes it is doable (but unusual).  First you need to tell Squid which requests 
should be rewritten, then send them to a rewrite program to be transformed.  
Identify the domains like this:

acl rewrite-domains dstdomain .yahoo.com .google.com etc)

Then set up a URL rewriting program, and only allow it to process requests 
matching the rewrite-domains ACL, like this:

url_rewrite_program /tmp/rewrite-program.pl
url_rewrite_extras "%>ru"
url_rewrite_access allow rewrite-domains
url_rewrite_access deny all

The program itself can be anything.  A very simple example in Perl might look 
like this:

#!/usr/bin/perl
use strict;
$| = 1;

# Enter loop
while (my $thisline = <>) {
my @parts = split(/\s+/, $thisline);
my $url = $parts[0];
$url =~ s/http:\/\/(.*)/http:\/\/192.168.1.1\/$1/g;
print "OK rewrite-url=\"$url\"\n";
}

If you input http://www.yahoo.com/page.html, this will be transformed to 
http://192.168.1.1/www.google.com/page.html.  The helper just needs to print 
that out prepended by "OK rewrite-url=xxx".  More info at 
http://www.squid-cache.org/Doc/config/url_rewrite_program/

Of course, you will need something listening on 192.168.1.1 (Apache, nginx, 
whatever) that can deal with those rewritten requests.  That is an unusual way 
of getting requests to 192.168.1.1 though, because you are effectively putting 
the hostname component into the URL then sending it to a web service and 
expecting it to deal with that.

Another note.  If you have a cache_peer defined, you might need some config to 
force rewritten requests to be sent to 192.168.1.1 and not your cache peer.  In 
that case this should do the trick:

acl rewrite-host dst 192.168.1.1
always_direct allow rewrite-host

HtH.

Luke


___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] Question about the url rewrite before proxy out

[Bill Yuan]

​Hello,

i am looking for a proxy which can "bounce" the request, which is not a
classic proxy.

I want it works in this way.
e.g. a proxy is running a 192.168.1.1
and when i want to open www.yahoo.com, i just need call
http://192.168.1.1/www.yahoo.com
the proxy can pickup the the host "www.yahoo.com" from the URI, and
retrieve the info for me​,

so it need to get the new $host from $location, and remove the $host from
the $location before proxy pass it.

it is doable via squid?

Regards
Bill
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users