Re: [squid-users] forwarded_for problems log client ip apache 2.4
Hey,
It is off-topic but I do have a setup that works with this and it
depends on couple things.
The first thing is that if it's not clear to me how you use the squid
and the apache services together.
You squid.conf shows two ports that both are in forward mode rather then
reverse mode which the setup would be pretty different by the proxy
functionality.
The basic scenario that the proxy provides a Forwarded-For header is
when it is when it has someone to inform about it such as internal
service or a reverse proxy.
When it's a parent or sibling proxy then the forwarded_for option should
be in "on" mode. Just notice that if you have some WAN connection in the
middle then without an HTTPS secured connection it would probably be
meaningless for a service unless it has a specific set of IP addresses
that it trusts.(unless the service has a reverse or forward dns
resolution mechanism that will "automatically" add\identify origin
sources by the domain name A\\CNAME records)
Currently squid doesn't have the option to use some ACLs in order to
decide to who\what he will send the forwarded-for headers which might be
important in use cases like I think yours is.
Basically based on the assumption that this proxy doesn't have any child
proxy services the right way to implement the forwarded-for is using the
"truncate" and not the "on" option to avoid any sort of ip
impersonations.(since any client can add "X-Forwarded-For: X.Y.Z.I" to
the request).
As for the apache remote_ip module and squid it is very simple to test,
a simple tcpdump on the proxy or the apache server with some filters
will show you what is on the wire and what the apache server receives.
The main question is what you do see in your apache logs and what you
expect to appear in them?
I can lend you my working remoteip modules settings:
RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy 192.168.10.10
From the docs at:
-
https://httpd.apache.org/docs/trunk/mod/mod_remoteip.html#remoteiptrustedproxy
-
https://httpd.apache.org/docs/trunk/mod/mod_remoteip.html#remoteipinternalproxy
I assume that you are wrongly using the
"RemoteIPTrustedProxy" directive to trust this proxy about internal 10/8
192.168/16 etc addresses spaces which it cannot(as documented).
So my suggestion is to try the "RemoteIPInternalProxy" instead of
"RemoteIPTrustedProxy".
Notice that remote_ip is IP related module and will not result in
reporting any sort of domain name in the access logs, resulting in such
log format will be an apache log related subject which I have never used.
Currently the log format I am using in apache is:
LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"
%v" combined_vhost
Which will show the remote_ip module resolved IP address and will report
the target vhost in the end of the log line so it won't break some log
parsing tools.
All The Bests,
Eliezer
* I wrote this long email partially as documentation of the subject for
later use in searches.
On 28/01/2016 15:38, L.P.H. van Belle wrote:
Hai,
I having some troubles to get my client ip (and/or hostname) logged in
my apache webserver.
I do think this is something in my squid setup, but i can find it..
So if anyone can help me out a bit, would be great.
I’ve tested with the forwarded_for options tried all options here.
http://www.squid-cache.org/Versions/v3/3.5/cfgman/forwarded_for.html
im using Debian Jessie, Apache 2.4 with mod_remoteip
http://httpd.apache.org/docs/current/mod/mod_remoteip.html#remoteipheader
My settings for remoteip ( and yes the modules is enabled )
a2query -m | grep remote
remoteip (enabled by site administrator)
# for remote proxy setup
RemoteIPHeader X-Forwarded-For
# for cluster setup
#RemoteIPHeader X-Real-IP
RemoteIPTrustedProxy 127.0.0.1/8
RemoteIPTrustedProxy 192.168.x.x/24
RemoteIPTrustedProxy 192.168.x.x/24
RemoteIPTrustedProxy prxy1.internal.domain.tld
RemoteIPTrustedProxy prxy2.internal.domain.tld
#original : LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\"
\"%{User-Agent}i\"" combined
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\""
combined
any tips on howto debug this, i did find lots of things with google, but
none worked for me.
This is my (sanitized) squid config, default values are not shown.
Any improvement tips are welkom ;-) but my bigest problem now is
getting the ip of the client in my webserver logs.
Greetz,
Louis
# squid 3.5.12 config
auth_param negotiate program /usr/lib/squid/negotiate_wrapper_auth -d \
--kerberos /usr/lib/squid/negotiate_kerberos_auth -s
HTTP/prxy1.internal.domain.tld@REALM \
--ntlm /usr/bin/ntlm_auth --helper-protocol=gss-spnego
--domain=NTDOMAIN
auth_param negotiate children 50 startup=10 idle=1
auth_param negotiate keep_alive on
auth_param basic program /usr/lib/squid/basic_ldap_auth -R \
-b
Re: [squid-users] forwarded_for
Hey Brian, Can you test this issue with the 3.5.x or 3.4.x RPM's I released? I have couple production servers running with 3.4 and 3.5 with truncate option to allow the backhand servers see the client IP. Eliezer * http://wiki.squid-cache.org/KnowledgeBase/CentOS On 11/06/2015 16:38, Snyder, Brian wrote: Hello All, I am running squid 3.3.8 on CentOS 7.1. The kernel is 3.10.0-229.4.2.el7.x86_64. I am having an issue where the forwarded_for directive is not working correctly in squid.conf. I initially started the server hiding the client IP using the delete setting. We have now changed direction and I wish to use the client IP instead of the proxy. However, no setting will change the server IP presented. I have even tried to reconfigure with the line completely removed from the config with no luck. Stopping the service does not help. Even a server reboot does nothing. I have thought about just doing a rebuild. Any help would be appreciated. Thanks, Brian ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] forwarded_for
On 12/06/2015 1:38 a.m., Snyder, Brian wrote: Hello All, I am running squid 3.3.8 on CentOS 7.1. The kernel is 3.10.0-229.4.2.el7.x86_64. I am having an issue where the forwarded_for directive is not working correctly in squid.conf. I initially started the server hiding the client IP using the delete setting. We have now changed direction and I wish to use the client IP instead of the proxy. However, no setting will change the server IP presented. I have even tried to reconfigure with the line completely removed from the config with no luck. Stopping the service does not help. Even a server reboot does nothing. I have thought about just doing a rebuild. Removing the directive entirely from your config should work. Note that the recipient server must be interpreting the X-Forwarded-For header *correctly* (as a reverse-path list of mixed IPv4, IPv6, and text labels - not just a single IP). Also any other middleware between your proxy and the server touching the header will affect the contents the origin server receives. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] forwarded_for ? in 3.2.x
On Apr 25, 2011, at 4:09 AM, Amos Jeffries wrote: On 22/04/11 02:08, jeffrey j donovan wrote: Greetings, I have a a transparent squid in a private net with a 1-1 NAT, Im trying to get a good understanding of what my clients look like to the outside. What is the Default setting for forwarded_for if my system is running intercept? forwarded_for on is the default for all modes. The client IP *as seen by Squid* is added to the header. to my understanding if I leave the X-Forwarded-For header my natted clients ip will be the visible requestor ? Whatever the client IP making the request was will be noted as the original requestor. The internal private IP ranges have no meaning to external viewers. They simply indicate that there was a NAT step. in the past did we strip that out or is it something new? Nothing has changed in Squid. Maybe your config or something outside Squid was playing with it. is there a way to have the final request return the global NAT ip of the client ? There is no such global IP for the client, at least for port 80. The client never touches the Internet when intercepted into Squid. This is one of the few benefits of interception. Squid box is the only public TCP/IP address touching the Internet. currently squid seems to be the final, i think. can someone clarify this option for me, thanks -j 192.168.1.2 --- 192.168.1.1[ squid]10.10.10.1 -- 10.10.10.2 [ IP NAT ] -- GLOBAL Correct. forwarded_for New setting options. transparent, truncate, delete. If set to transparent, Squid will not alter the X-Forwarded-For header in any way. If set to delete, Squid will delete the entire X-Forwarded-For header. If set to truncate, Squid will remove all existing X-Forwarded-For entries, and place itself as the sole entry. ... as you cut-n-pasted from the documentation, that is what it does. The place itself as the sole entry was incorrect. Fixed in recent releases to be place the client IP as the sole entry Going back to your initial goal get a good understanding of what my clients look like to the outside... The outside all sees Squid global IP connecting to them and making requests. For smart web services that attempt to use advanced transfer features they see the Via: header indicating the client and Squid capabilities so nothing breaks halfway back. For smart security systems that attempt IP-based security (the ones that do it well anyway) they see the X-Forwarded-For header with a group of identifiers that can be combined to classify different end clients apart. Amos thanks for the clarity :) btw 3.2.12 build on Darwin ppc/intel works great. -j
Re: [squid-users] forwarded_for ? in 3.2.x
On 22/04/11 02:08, jeffrey j donovan wrote: Greetings, I have a a transparent squid in a private net with a 1-1 NAT, Im trying to get a good understanding of what my clients look like to the outside. What is the Default setting for forwarded_for if my system is running intercept? forwarded_for on is the default for all modes. The client IP *as seen by Squid* is added to the header. to my understanding if I leave the X-Forwarded-For header my natted clients ip will be the visible requestor ? Whatever the client IP making the request was will be noted as the original requestor. The internal private IP ranges have no meaning to external viewers. They simply indicate that there was a NAT step. in the past did we strip that out or is it something new? Nothing has changed in Squid. Maybe your config or something outside Squid was playing with it. is there a way to have the final request return the global NAT ip of the client ? There is no such global IP for the client, at least for port 80. The client never touches the Internet when intercepted into Squid. This is one of the few benefits of interception. Squid box is the only public TCP/IP address touching the Internet. currently squid seems to be the final, i think. can someone clarify this option for me, thanks -j 192.168.1.2 --- 192.168.1.1[ squid]10.10.10.1 -- 10.10.10.2 [ IP NAT ] -- GLOBAL Correct. forwarded_for New setting options. transparent, truncate, delete. If set to transparent, Squid will not alter the X-Forwarded-For header in any way. If set to delete, Squid will delete the entire X-Forwarded-For header. If set to truncate, Squid will remove all existing X-Forwarded-For entries, and place itself as the sole entry. ... as you cut-n-pasted from the documentation, that is what it does. The place itself as the sole entry was incorrect. Fixed in recent releases to be place the client IP as the sole entry Going back to your initial goal get a good understanding of what my clients look like to the outside... The outside all sees Squid global IP connecting to them and making requests. For smart web services that attempt to use advanced transfer features they see the Via: header indicating the client and Squid capabilities so nothing breaks halfway back. For smart security systems that attempt IP-based security (the ones that do it well anyway) they see the X-Forwarded-For header with a group of identifiers that can be combined to classify different end clients apart. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.7 and 3.1.12.1
Re: [squid-users] forwarded_for
On Mon, 6 Sep 2004, Dmitry Shukaylo wrote: Hello! Is it possible to apply 'forwarded_for' directive on acl, like: acl somewhere_someone dst ip-address/netmask forwarded_for allow somewhere_someone No, but patches implementing this feature to control when forwarded_for is to be used is welcome. It is just a matter of getting the implementation done, nothing really prevents this from getting done except time and the fact that none of the customers with an commercial Squid support contract have asked for this. Regards Henrik
Re: [squid-users] forwarded_for
Is it possible to apply 'forwarded_for' directive on acl, like: acl somewhere_someone dst ip-address/netmask forwarded_for allow somewhere_someone M We can do this with tcp_outgoing_aaddress TAG Unfortunately tcp_outgoing_address TAG does not approach He serves absolutely for other purposes. It is necessary, that (for predefined destination addresses only), HTTP protocol was substituted with kludge of the source address, that does forwarded_for... For example: user (10.0.0.1) - GW - target_service (100.200.0.1 : 80) comes as GW, but X_HTTP_FORWARDED_FOR = 10.0.0.1 user (10.10.0.1) - GW - other_service (200.100.0.1 : 80) comes as GW and X_HTTP_FORWARDED_FOR = unknown where GW - means gateway where squid is installed -- Dmitry Shukaylo mailto:[EMAIL PROTECTED]
Re: [squid-users] forwarded_for
Is it possible to apply 'forwarded_for' directive on acl, like: acl somewhere_someone dst ip-address/netmask forwarded_for allow somewhere_someone We can do this with tcp_outgoing_aaddress TAG - Muthu --- === It is a Virus Free Mail === Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.752 / Virus Database: 503 - Release Date: 9/3/2004
Re: [squid-users] forwarded_for
Duane Wessels wrote: Here is how X-Forwarded-For works: Each proxy in the hierarchy is going to append something to the X-Forwarded-For header. If 'forwarded_for' is on, then Squid appends the client's IP address. Yes, it works in our first level of proxys. If it is off, then Squid appends the string 'unknown'. Here, in our third level of proxys fail. with forwarded_for off appear the ips of clients instead the string unknown Thanks Duane and Sorry if I am something confused. Emilio. smime.p7s Description: S/MIME Cryptographic Signature
Re: [squid-users] forwarded_for
On Fri, 12 Dec 2003, Emilio Casbas wrote: Duane Wessels wrote: Here is how X-Forwarded-For works: Each proxy in the hierarchy is going to append something to the X-Forwarded-For header. If 'forwarded_for' is on, then Squid appends the client's IP address. Yes, it works in our first level of proxys. If it is off, then Squid appends the string 'unknown'. Here, in our third level of proxys fail. with forwarded_for off appear the ips of clients instead the string unknown I don't think there are any bugs with the 'forwarded_for' directive. Perhaps you have the directive repeated in your config file and it is really set to on when you think it is off? You can request 'config' from the cache manager and see what Squid has the value set to internally. Duane W.
Re: [squid-users] forwarded_for
See squid.conf or the FAQ. Regards Henrik On Fri, 12 Dec 2003, Emilio Casbas wrote: Hi, In one jerarquia of 3 levels of proxy-cache, we have configured so that the first level shows ips of the clients (forwarded on) and works well, and in proxis-cache of third-level of exit (forwarded off), but here is where it does not work and shows the following thing: X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x where x.x.x.x is ips of our clients. instead of: X-Forwarded-For: unknown The squid version is squid-2.5Stable4 Thanks in advanced. Emilio
Re: [squid-users] forwarded_for
Duane Wessels wrote: On Fri, 12 Dec 2003, Emilio Casbas wrote: Duane Wessels wrote: Here is how X-Forwarded-For works: Each proxy in the hierarchy is going to append something to the X-Forwarded-For header. If 'forwarded_for' is on, then Squid appends the client's IP address. Yes, it works in our first level of proxys. If it is off, then Squid appends the string 'unknown'. Here, in our third level of proxys fail. with forwarded_for off appear the ips of clients instead the string unknown I don't think there are any bugs with the 'forwarded_for' directive. Perhaps you have the directive repeated in your config file and it is really set to on when you think it is off? You can request 'config' from the cache manager and see what Squid has the value set to internally. This is from cache manager: tcp_recv_bufsize 0 bytes err_html_text memory_pools on memory_pools_limit 0 bytes forwarded_for off --- log_icp_queries on icp_hit_stale off minimum_direct_hops 4 I don't know what it can be happening. Duane W. Thanks. Emilio. smime.p7s Description: S/MIME Cryptographic Signature
Re: [squid-users] forwarded_for
On Fri, 2003-12-12 at 12:47, Emilio Casbas wrote: Duane Wessels wrote: On Fri, 12 Dec 2003, Emilio Casbas wrote: Duane Wessels wrote: Here is how X-Forwarded-For works: Each proxy in the hierarchy is going to append something to the X-Forwarded-For header. If 'forwarded_for' is on, then Squid appends the client's IP address. [snip] Here, in our third level of proxys fail. with forwarded_for off appear the ips of clients instead the string unknown [snip] Are you expecting it to OVERWRITE (sobre-escribir) instead of APPEND (agregar)? As Duane said, Squid does not remove or replace X-Forwarded-For entries, it only adds to them. If you want to remove the header completely, use the 'header_access' and 'header_replace' directives.
Re: [squid-users] forwarded_for
On Fri, 12 Dec 2003, Emilio Casbas wrote: Here, in our third level of proxys fail. with forwarded_for off appear the ips of clients instead the string unknown Then forwarded_for is enabled. One thing you can do if making sure forwarded_for is off on all child caches is to use http_header_access on the main proxy to deny the header from being forwarded. Regards Henrik
RE: [squid-users] forwarded_for
Hi Emilio, This is my outbound squid config located on my firewall - which is the last squid in the chain configured as follows utilizing header_access filtering, # # header filtering header_access Allow allow all header_access Authorization allow all header_access WWW-Authenticate allow all header_access Cache-Control allow all header_access Content-Encoding allow all header_access Content-Length allow all header_access Content-Type allow all header_access Date allow all header_access Expires allow all header_access Host allow all header_access If-Modified-Since allow all header_access Last-Modified allow all header_access Location allow all header_access Pragma allow all header_access Accept-Charset allow all header_access Accept-Encoding allow all header_access Accept-Language allow all header_access Content-Language allow all header_access Mime-Version allow all header_access Retry-After allow all header_access Title allow all header_access Connection allow all header_access Proxy-Connection allow all header_access Set-Cookie allow all header_access Cookie allow all header_access Accept allow all header_access User-Agent allow all header_access Referer allow all header_access All deny all # Which removes all the ugly stuff!. ;-) -david -Original Message- From: Henrik Nordstrom [mailto:[EMAIL PROTECTED] Sent: Saturday, 13 December 2003 04:47 To: Emilio Casbas Cc: [EMAIL PROTECTED] Subject: Re: [squid-users] forwarded_for See squid.conf or the FAQ. Regards Henrik On Fri, 12 Dec 2003, Emilio Casbas wrote: Hi, In one jerarquia of 3 levels of proxy-cache, we have configured so that the first level shows ips of the clients (forwarded on) and works well, and in proxis-cache of third-level of exit (forwarded off), but here is where it does not work and shows the following thing: X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x X-Forwarded-For: x.x.x.x, x.x.x.x where x.x.x.x is ips of our clients. instead of: X-Forwarded-For: unknown The squid version is squid-2.5Stable4 Thanks in advanced. Emilio
Re: [squid-users] forwarded_for off has no effect on Squid 2.5 Stable 3 CVS ( Windows NT)?
Hi Markus, At 14.32 18/08/2003, Wagner Markus wrote: Hello, today I added the line: forwarded_off in my squid.conf file but I when I go to http://devel.squid-cache.org/cgi-bin/test I still see my ip address after HTTP_X_FORWARDED_FOR instead of unknown. Is this a known bug? How can I disable this header? thanks, Markus Wagner It seems a generic Squid bug, I have just applied a little patch to my Squid and now it works fine. Please open a Bug on Bugzilla (http://www.squid-cache.org/bugs/index.cgi). Regards Guido - Guido Serassio Acme Consulting S.r.l. Via Gorizia, 69 10136 - Torino - ITALY Tel. : +39.011.3249426 Fax. : +39.011.3293665 Email: [EMAIL PROTECTED] WWW: http://www.acmeconsulting.it/
RE: [squid-users] forwarded_for off has no effect on Squid 2.5 Stable 3 CVS ( Windows NT)?
today I added the line: forwarded_off in my squid.conf file but I when I go to http://devel.squid-cache.org/cgi-bin/test I still see my ip address after HTTP_X_FORWARDED_FOR instead of unknown. Is this a known bug? How can I disable this header? Maybe a Typo? The option should read forwarded_for off Hermann
RE: [squid-users] forwarded_for off has no effect on Squid 2.5Stable 3 CVS ( Windows NT)?
no - it was a typo in my mail (but it is ok in the subject of the mail and in the conf file) Well - I am using now: header_access X-Forwarded-For deny all I also will open a Bug as Guido suggested Markus -Ursprüngliche Nachricht- Von: Hermann Strassner [mailto:[EMAIL PROTECTED] Gesendet: Montag, 18. August 2003 15:32 An: Wagner Markus Cc: Squid Mailinglist Betreff: RE: [squid-users] forwarded_for off has no effect on Squid 2.5 Stable 3 CVS ( Windows NT)? today I added the line: forwarded_off in my squid.conf file but I when I go to http://devel.squid-cache.org/cgi-bin/test I still see my ip address after HTTP_X_FORWARDED_FOR instead of unknown. Is this a known bug? How can I disable this header? Maybe a Typo? The option should read forwarded_for off Hermann
