Re: [squid-users] Content Encoding Error
On Thursday, May 09, 2013 09:18:53 PM Amos Jeffries wrote: On 10/05/2013 2:24 a.m., cac...@quantum-sci.com wrote: OK I guess I have to ditch Squid. I can't live with this. Well, if you cant or dont want to supply the information needed to help there is very little help possible. LOL, it appears you feel I am being uncooperative. What does your squid.conf contain? http://pastebin.com/ke5WQkdj This is the squid.conf documentation with some lines uncommented. Notably the _incomplete example_ of how to do anonymous proxy configuration for HTTP/1.0 traffic. Few of the HTTP/1.1 headers are handled there. The error you report is usually seen when Accept*, Content-Encoding or Tansfer-Encoding are screwed with. The logs you wont supply would have shown which was the problem. 'The logs I -won't- supply'? You mean the logs I -can't- supply because Squid hasn't made logs for me for months. The squid.conf documentation? That is exactly what comes with Debian's squid package. What do you mean documentation? Where is the secret place that you are getting the real .conf file? _incomplete example_ ? Where is the secret place that you get the full header anonymization?
Re: [squid-users] Content Encoding Error
On 11/05/2013 12:58 a.m., cac...@quantum-sci.com wrote: On Thursday, May 09, 2013 09:18:53 PM Amos Jeffries wrote: On 10/05/2013 2:24 a.m., cac...@quantum-sci.com wrote: OK I guess I have to ditch Squid. I can't live with this. Well, if you cant or dont want to supply the information needed to help there is very little help possible. LOL, it appears you feel I am being uncooperative. What does your squid.conf contain? http://pastebin.com/ke5WQkdj This is the squid.conf documentation with some lines uncommented. Notably the _incomplete example_ of how to do anonymous proxy configuration for HTTP/1.0 traffic. Few of the HTTP/1.1 headers are handled there. The error you report is usually seen when Accept*, Content-Encoding or Tansfer-Encoding are screwed with. The logs you wont supply would have shown which was the problem. 'The logs I -won't- supply'? You mean the logs I -can't- supply because Squid hasn't made logs for me for months. cant or wont. I wasn't sure. Not everyone using those anonymous configs are willing to post the details here in public. If you like we'd probably get that sorted. I'm thinking its a permissions issue in the logs directory, overflowing logs due to log rotation errors (ALL,3 can output a lot of data and get into a bit of trouble getting past 2 or 4 GB). The squid.conf documentation? That is exactly what comes with Debian's squid package. What do you mean documentation? Where is the secret place that you are getting the real .conf file? With the squid3 packages you will find it in /etc/squid3/squid.conf. With the file you posted at squid.conf.documented. if you are building your own and installign over an existing Squid, you will find the new default config in squid.conf.default next to your squid.conf and an updated documentation file at squid.conf.documented. _incomplete example_ ? Where is the secret place that you get the full header anonymization? There isn't one published that I'm aware of. Its just that nobody has updated that one in most of a decade to allow the more recently created required headers through. Like you are probably encountering errors due to Transfer-Encoding and TE being missing. Amos
Re: [squid-users] ftp directory listing fails for win7/ie8
On 10/05/2013 4:10 p.m., Brett Lymn wrote: I am using squid v3.3.2 and have found that ftp directory listings don't seem to be working for windows 7 with IE 8 but they work fine with firefox. When using IE 8 and visiting the url ftp://ftp.netbsd.org/ our parent proxy returns a ERR_DIR_LISTING error but if we use firefox from the same machine the directory listing works fine. ERR_DIR_LISTING is the directory listing page template name. I think Squid is sending the directory listing, but for some reason IE is not displaying it. Performing a sniff on the traffic from the parent proxy shows the directory listing being retrieved fine when either browser is used but for some reason it delivers an error to IE8. Please check your friendly error pages setting in IE is disabled. Also, check your use HTTP/1.1 with proxies setting in IE is enabled. It would also be worthwhile locating the HTTP status codes Squid is sending to the browser with the ERR_DIR_LISTING page. Amos
Re: [squid-users] Pre-populating web cache?
On 10/05/2013 2:41 p.m., Yang Zhang wrote: I have a database of HTTP requests and responses which I'd like to import into Squid so it can serve from this cache. Is there any documentation/resources describing the data organization/layout of a Squid cache directory, plus any gotchas/pitfalls I may want to be aware of? Thanks. It is certainly possible with any of the caches. There are some gotchas: 1) Squid must not be using the cache at the time new content is added, 2) you must remove any swap.state journal (may be inside one of the storage system entries). 3) Squid may drop your added content if its too old by the time its needed. Unfortunately I'm not sure if the formats are documented clearly anywhere. Its about time they were so I will have a look around for you and look into writing some if necessary. HTH Amos
Re: [squid-users] Content Encoding Error
On Friday, May 10, 2013 06:17:10 AM Amos Jeffries wrote: If you like we'd probably get that sorted. I'm thinking its a permissions issue in the logs directory, overflowing logs due to log rotation errors (ALL,3 can output a lot of data and get into a bit of trouble getting past 2 or 4 GB). OK I've always gone to /var/log/squid, which is empty, but I see there is now a squid3. Logs are there, although don't seem to be getting rotated. With the squid3 packages you will find it in /etc/squid3/squid.conf. With the file you posted at squid.conf.documented. if you are building your own and installign over an existing Squid, you will find the new default config in squid.conf.default next to your squid.conf and an updated documentation file at squid.conf.documented. Actually that is the squid.conf you get in /etc/squid3 when installing squid3 in Debian. That extensively-commented .conf is what we've always gotten there in Debian. There isn't one published that I'm aware of. Its just that nobody has updated that one in most of a decade to allow the more recently created required headers through. Like you are probably encountering errors due to Transfer-Encoding and TE being missing. So the problem is new headers. I added request_header_access Transfer-Encoding allow all reply_header_access Transfer-Encoding allow all ... and it fixed it, thanks. It is worrying though that this is not being kept up, and anonymized headers is not documented. That we have more of this to look forward to. Why are ppl afraid to post what they have? It's not like we're going to hack in to their squid server via anonymous headers.
Re: [squid-users] logging issues
On 9/05/2013 12:27 a.m., Mr J Potter wrote: Works for me! A few notes for anyone who needs them below. And some extra notes in case anyone tries to use them ;-) Thanks again everyone. Jim UK Issues/gotchas: It doesn't work behind parent proxies. Well, it requires the parent proxy to be SSL-enabled as well. It works with NTLM and ident You need your own certificate authority on all clients. To build squid3.2 on debian 7: dependencies: install everything so you can build squid3.1 from source get squid 3.2 source and build with: ./configure \ --prefix=/srv/squid32 \ --sysconfdir=/srv/squid32/conf \ --localstatedir=/srv/squid32/var \ --enable-auth \ --enable-auth-ntlm=SSPI,smb_lm \ --enable-ssl \ --enable-ssl-crtd \ --enable-icap-client Or better grab the 3.3 source package from Sid/Unstable repositories. Add the SSL options above to debian/rules and build. You will also need to install the libssl-dev package for those new options to work. The 3.3 sources will require some adjustments to the rest of these notes as follows... Follow instructions on creating a CA from: http://www.mydlp.com/how-to-configure-squid-3-2-ssl-bumping-dynamic-ssl-certificate-generation/ Here's my config cache_effective_user proxy #cache_peer caffreys.bristol-cyps.org.ukparent3128 3130 default cache_peer courage.bristol-cyps.org.uk parent3128 3130 default #no-delay #no-query no-digest no-netdb-exchange ## default #cache_peer_access caffreys.bristol-cyps.org.uk allow all cache_peer_access courage.bristol-cyps.org.uk allow all forwarded_for off url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf #auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp #auth_param ntlm children 20 startup=0 idle=1 #acl authdUsers proxy_auth REQUIRED acl authdUsers ident REQUIRED acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl HTTPS proto HTTPS acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT ## HTTPS busting bit!!! ssl_bump allow all You probably want to adjust this as you go to add exceptions as you find sites which do not work with bumping or non-HTTPS traffic using CONNECT. NOTE: with 3.3 sources you can use server-first instead of allow for a more user-friendly bump. sslproxy_cert_error allow all NOTE: using allow all with sslproxy_cert_error enables any hijacked SSL site to deliver content to your users without validation getting in the way and doing such things as warning them (or you) about the hijack. This is a very *BAD* idea to be this broad even with the 3.2 configuration. PS. The 3.3 releases server-first feature prevents it being necessary almost all of the time, and has some built-in defaults for silencing the harmless errors. # Or may be deny all according to your company policy # sslproxy_cert_error deny all sslproxy_flags DONT_VERIFY_PEER This is pretty much the same as allow all on cert errors. If you find yourself needing this *at all*, then what you actually need to do is a) update your ca-certificates package, that may involve adding the failing servers CA manually to openssl library configuration. see openssl documentation for more. sslcrtd_program /srv/squid32/libexec/ssl_crtd -s /srv/squid32/var/lib/ssl_db -M 4MB sslcrtd_children 5 # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on localhost is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow authdUsers http_access allow localnet http_access allow localhost # And finally deny all other access to this proxy http_access allow all NOTE: read the above comment. always_direct allow HTTPS NOTE: the above line disables using the cache_peer for any traffic. Squid-3.2 to 3.4 (at least) do not support
Re: [squid-users] Content Encoding Error
On 11/05/2013 1:34 a.m., cac...@quantum-sci.com wrote: On Friday, May 10, 2013 06:17:10 AM Amos Jeffries wrote: If you like we'd probably get that sorted. I'm thinking its a permissions issue in the logs directory, overflowing logs due to log rotation errors (ALL,3 can output a lot of data and get into a bit of trouble getting past 2 or 4 GB). OK I've always gone to /var/log/squid, which is empty, but I see there is now a squid3. Logs are there, although don't seem to be getting rotated. Aha. Then /etc/logrotate.d/squid (or .../squid3) probably needs to be updated for the current logs path. With the squid3 packages you will find it in /etc/squid3/squid.conf. With the file you posted at squid.conf.documented. if you are building your own and installign over an existing Squid, you will find the new default config in squid.conf.default next to your squid.conf and an updated documentation file at squid.conf.documented. Actually that is the squid.conf you get in /etc/squid3 when installing squid3 in Debian. That extensively-commented .conf is what we've always gotten there in Debian. Ar. I'm going to have to nudge Luigi about that again then. There isn't one published that I'm aware of. Its just that nobody has updated that one in most of a decade to allow the more recently created required headers through. Like you are probably encountering errors due to Transfer-Encoding and TE being missing. So the problem is new headers. I added request_header_access Transfer-Encoding allow all reply_header_access Transfer-Encoding allow all ... and it fixed it, thanks. It is worrying though that this is not being kept up, and anonymized headers is not documented. That we have more of this to look forward to. Why are ppl afraid to post what they have? It's not like we're going to hack in to their squid server via anonymous headers. It does let the boogymen know whats being allowed through though ;-). Who knows, not me. Amos
Re: [squid-users] Content Encoding Error
Hallo, Cacook, Du meintest am 10.05.13: If you like we'd probably get that sorted. I'm thinking its a permissions issue in the logs directory, overflowing logs due to log rotation errors (ALL,3 can output a lot of data and get into a bit of trouble getting past 2 or 4 GB). OK I've always gone to /var/log/squid, which is empty, but I see there is now a squid3. Logs are there, although don't seem to be getting rotated. Maybe a debian error. Rotating ist mostly a job for logrotate, and the most config files are in /etc/logrotate.d. What tells squid -v about sysconfdir (where squid.conf is found) and about with- logdir? What tells grep log sysconfdir/squid.conf about the logging directives? Viele Gruesse! Helmut
Re: [squid-users] Content Encoding Error
Ah yes, the logrotate path was wrong. Fixed it now. On Friday, May 10, 2013 07:06:00 AM Helmut Hullen wrote: What tells squid -v about sysconfdir (where squid.conf is found) and about with- logdir? --sysconfdir=/etc/squid3 --with-logdir=/var/log/squid3 What tells grep log sysconfdir/squid.conf about the logging directives? The only line uncommented: logfile_rotate 2 Hm, it appears that squid has built-in log rotate? Wouldn't the system logrotate interfere?
Re: [squid-users] FTP
Wow, even FTP works again. Can anyone recommend a secure FTP addon for Firefox? Rather than just having an HTML FTP listing I'd like it to automatically be an FTP client with drag and drop, bulk copy/move, etc.
RE: [squid-users] FTP
-Original Message- From: cac...@quantum-sci.com [mailto:cac...@quantum-sci.com] Sent: Friday, May 10, 2013 11:08 AM To: squid-users@squid-cache.org Subject: Re: [squid-users] FTP Wow, even FTP works again. Can anyone recommend a secure FTP addon for Firefox? Rather than just having an HTML FTP listing I'd like it to automatically be an FTP client with drag and drop, bulk copy/move, etc. FireFTP addon work well for everyone I've recommended it to. Jason This message has been scanned for malware by Websense. www.websense.com
Re: [squid-users] Content Encoding Error
Hallo, Cacook, Du meintest am 10.05.13: What tells grep log sysconfdir/squid.conf about the logging directives? The only line uncommented: logfile_rotate 2 And nothing like access_log stdio:/var/log/squid3/access.log Strange. Hm, it appears that squid has built-in log rotate? Wouldn't the system logrotate interfere? That depends! In my configurations logrotate works earlier than the squid routine. Viele Gruesse! Helmut
Re: [squid-users] Re: Re: Re: Memory error with squid 3.3.4 and kerberos authentication
Markus, I tested a few times, and looks like the error occurs when one first time i pass the testu...@my.domain.com to ext_kerberos_ldap_group_acl, and then, following, i pass one simple username, like testuser. That second time comes with the error, and the Domain: part of the log comes with garbage. At the other error i showed in previous message, it came empty, but look at this two other errors: testuser kerberos_ldap_group.cc(429): pid=6430 :2013/05/10 12:31:42| kerberos_ldap_group: INFO: Got User: testuser set default domain: »a kerberos_ldap_group.cc(434): pid=6430 :2013/05/10 12:31:42| kerberos_ldap_group: INFO: Got User: testuser Domain: »a *** glibc detected *** /usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk(): invalid pointer: 0x00611c90 *** === Backtrace: = /lib64/libc.so.6[0x36a7475916] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa] /lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9] === Memory map: 0040-00411000 r-xp fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 0061-00611000 rw-p 0001 fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 00611000-0164b000 rw-p 00:00 0 [heap] 31a420-31a4207000 r-xp fd:00 10375 /lib64/librt-2.12.so 31a4207000-31a4406000 ---p 7000 fd:00 10375 /lib64/librt-2.12.so 31a4406000-31a4407000 r--p 6000 fd:00 10375 /lib64/librt-2.12.so testuser kerberos_ldap_group.cc(429): pid=6361 :2013/05/10 12:30:05| kerberos_ldap_group: INFO: Got User: testuser set default domain: LOG_USER kerberos_ldap_group.cc(434): pid=6361 :2013/05/10 12:30:05| kerberos_ldap_group: INFO: Got User: testuser Domain: LOG_USER *** glibc detected *** /usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk(): invalid pointer: 0x00611ab0 *** === Backtrace: = /lib64/libc.so.6[0x36a7475916] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa] /lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9] === Memory map: 0040-00411000 r-xp fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 0061-00611000 rw-p 0001 fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 00611000-0164a000 rw-p 00:00 0 [heap] 31a420-31a4207000 r-xp fd:00 10375 /lib64/librt-2.12.so 31a4207000-31a4406000 ---p 7000 fd:00 10375 /lib64/librt-2.12.so 31a4406000-31a4407000 r--p 6000 fd:00 10375 /lib64/librt-2.12.so In production, I assume the error appears when one user authenticate with kerberos and then one another authenticate with basic. On Thu, May 9, 2013 at 10:07 PM, Carlos Defoe carlosde...@gmail.com wrote: Hello Markus, That was from testing with gdb, so it was not properly basic authentication, but a simple username (without kerberos nor netbios domain), given to ext_kerberos_ldap_group_acl. I will search in the cache logs for messages llike that, but i don't think i was running the helper with debug options inside squid. By the way, today i changed the auth to negotiate wrapper, now using kerberos and ntlm, before the basic auth. If the memory problem is from usernames received from basic authentication, i should see less errors now. I also made one isolated clone of the proxy, for testing. I will do more tests tomorrow. On Thu, May 9, 2013 at 7:57 PM, Markus Moeller hua...@moeller.plus.com wrote: Hi Carlos, The code will log INFO: Got User: %s set default domain: %s if the user has no domain and the default domain is used or INFO: Got User: %s Domain: NULL if the user has no domain and no default domain is provided. If NTLM is used the code will log INFO: Got User: %s Netbios Name: %s So I do not see a case where The following would be logged INFO: Got User: testusername Domain:.Can you get from the cache log the user authentication details (e.g. was it basic or kerberos or ntlm) ? Markus Markus Moeller hua...@moeller.plus.com wrote in message news:kmfc9i$80t$1...@ger.gmane.org... I will check the code to see if there is something wrong if basic auth users hitting the helper. Markus Carlos Defoe carlosde...@gmail.com wrote in message news:CAHsHsyt0F=l9bpng5kvqmvrvijhyyxacqwwyx4dsqapasao...@mail.gmail.com... I'm already using -D MYREALM.COM. The complete command line is: ext_kerberos_ldap_group_acl -g group4acc...@myrealm.com -D MYREALM.COM So it could work with kerberos and basic athenticated users. I also tried adding -N MYREALM:MYREALM.COM to match the netbios name of the domain, but it didn't seem to make any difference. I think the Domain in the line kerberos_ldap_group: INFO: Got User: testusername Domain: means the domain if the username is given like
[squid-users] Like HipHop and Squid Proxy? Ok, why not have both.
I know this is a bit off topic but what the heck, heres some hiphop for squid users. A kid on twitter sent me this link asking me to listen to it, he says squid so much in his raps, Ive taken to it while working on my squid box. Album here. I think any squid fanatic whos also open to hip hop would like that too. http://izzyjonesmusic.bandcamp.com/album/squid - Signed, Fix Nichols http://www.squidblacklist.org
[squid-users] Re: Re: Re: Re: Memory error with squid 3.3.4 and kerberos authentication
Hi Carlos, Could you run ext_kerberos_ldap_group_acl with -d and sent me the whole output please ? It looks like a variable has not been freed and set to NULL, but I can't see it in the code. Thank you Markus Carlos Defoe carlosde...@gmail.com wrote in message news:CAHsHsyvWT_AT4ZHFQ8Kv1QfPQeiKG1xzVLazZDYK=56g30=p...@mail.gmail.com... Markus, I tested a few times, and looks like the error occurs when one first time i pass the testu...@my.domain.com to ext_kerberos_ldap_group_acl, and then, following, i pass one simple username, like testuser. That second time comes with the error, and the Domain: part of the log comes with garbage. At the other error i showed in previous message, it came empty, but look at this two other errors: testuser kerberos_ldap_group.cc(429): pid=6430 :2013/05/10 12:31:42| kerberos_ldap_group: INFO: Got User: testuser set default domain: »a kerberos_ldap_group.cc(434): pid=6430 :2013/05/10 12:31:42| kerberos_ldap_group: INFO: Got User: testuser Domain: »a *** glibc detected *** /usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk(): invalid pointer: 0x00611c90 *** === Backtrace: = /lib64/libc.so.6[0x36a7475916] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa] /lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9] === Memory map: 0040-00411000 r-xp fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 0061-00611000 rw-p 0001 fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 00611000-0164b000 rw-p 00:00 0 [heap] 31a420-31a4207000 r-xp fd:00 10375 /lib64/librt-2.12.so 31a4207000-31a4406000 ---p 7000 fd:00 10375 /lib64/librt-2.12.so 31a4406000-31a4407000 r--p 6000 fd:00 10375 /lib64/librt-2.12.so testuser kerberos_ldap_group.cc(429): pid=6361 :2013/05/10 12:30:05| kerberos_ldap_group: INFO: Got User: testuser set default domain: LOG_USER kerberos_ldap_group.cc(434): pid=6361 :2013/05/10 12:30:05| kerberos_ldap_group: INFO: Got User: testuser Domain: LOG_USER *** glibc detected *** /usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk(): invalid pointer: 0x00611ab0 *** === Backtrace: = /lib64/libc.so.6[0x36a7475916] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa] /lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd] /usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9] === Memory map: 0040-00411000 r-xp fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 0061-00611000 rw-p 0001 fd:00 150231 /usr/local/squid/libexec/ext_kerberos_ldap_group_acl 00611000-0164a000 rw-p 00:00 0 [heap] 31a420-31a4207000 r-xp fd:00 10375 /lib64/librt-2.12.so 31a4207000-31a4406000 ---p 7000 fd:00 10375 /lib64/librt-2.12.so 31a4406000-31a4407000 r--p 6000 fd:00 10375 /lib64/librt-2.12.so In production, I assume the error appears when one user authenticate with kerberos and then one another authenticate with basic. On Thu, May 9, 2013 at 10:07 PM, Carlos Defoe carlosde...@gmail.com wrote: Hello Markus, That was from testing with gdb, so it was not properly basic authentication, but a simple username (without kerberos nor netbios domain), given to ext_kerberos_ldap_group_acl. I will search in the cache logs for messages llike that, but i don't think i was running the helper with debug options inside squid. By the way, today i changed the auth to negotiate wrapper, now using kerberos and ntlm, before the basic auth. If the memory problem is from usernames received from basic authentication, i should see less errors now. I also made one isolated clone of the proxy, for testing. I will do more tests tomorrow. On Thu, May 9, 2013 at 7:57 PM, Markus Moeller hua...@moeller.plus.com wrote: Hi Carlos, The code will log INFO: Got User: %s set default domain: %s if the user has no domain and the default domain is used or INFO: Got User: %s Domain: NULL if the user has no domain and no default domain is provided. If NTLM is used the code will log INFO: Got User: %s Netbios Name: %s So I do not see a case where The following would be logged INFO: Got User: testusername Domain:.Can you get from the cache log the user authentication details (e.g. was it basic or kerberos or ntlm) ? Markus Markus Moeller hua...@moeller.plus.com wrote in message news:kmfc9i$80t$1...@ger.gmane.org... I will check the code to see if there is something wrong if basic auth users hitting the helper. Markus Carlos Defoe carlosde...@gmail.com wrote in message news:CAHsHsyt0F=l9bpng5kvqmvrvijhyyxacqwwyx4dsqapasao...@mail.gmail.com... I'm already using -D MYREALM.COM. The complete command line is: ext_kerberos_ldap_group_acl -g group4acc...@myrealm.com -D MYREALM.COM So it could work with kerberos and basic athenticated users. I also tried adding -N
Re: [squid-users] Pre-populating web cache?
How does it look a database of queries and responses? You mean, you have the html, imgs and all the linked stuff stored on disk? On Fri, May 10, 2013 at 10:30 AM, Amos Jeffries squ...@treenet.co.nz wrote: On 10/05/2013 2:41 p.m., Yang Zhang wrote: I have a database of HTTP requests and responses which I'd like to import into Squid so it can serve from this cache. Is there any documentation/resources describing the data organization/layout of a Squid cache directory, plus any gotchas/pitfalls I may want to be aware of? Thanks. It is certainly possible with any of the caches. There are some gotchas: 1) Squid must not be using the cache at the time new content is added, 2) you must remove any swap.state journal (may be inside one of the storage system entries). 3) Squid may drop your added content if its too old by the time its needed. Unfortunately I'm not sure if the formats are documented clearly anywhere. Its about time they were so I will have a look around for you and look into writing some if necessary. HTH Amos
[squid-users] Rate limiting for certain servers
Can I throttle the request rate to a certain set of servers? I came across this but it seems to only be for throttling specific clients: http://wiki.squid-cache.org/Features/DelayPools (The server is an API and we'd like to respect its rate limit among requests from our various client processes.) Thanks.
Re: [squid-users] Pre-populating web cache?
On Fri, May 10, 2013 at 12:06 PM, Carlos Defoe carlosde...@gmail.com wrote: How does it look a database of queries and responses? You mean, you have the html, imgs and all the linked stuff stored on disk? These are actually just JSON API requests/responses, so there is no html/images/etc.
Re: [squid-users] Pre-populating web cache?
On Fri, May 10, 2013 at 6:30 AM, Amos Jeffries squ...@treenet.co.nz wrote: On 10/05/2013 2:41 p.m., Yang Zhang wrote: I have a database of HTTP requests and responses which I'd like to import into Squid so it can serve from this cache. Is there any documentation/resources describing the data organization/layout of a Squid cache directory, plus any gotchas/pitfalls I may want to be aware of? Thanks. It is certainly possible with any of the caches. There are some gotchas: 1) Squid must not be using the cache at the time new content is added, 2) you must remove any swap.state journal (may be inside one of the storage system entries). 3) Squid may drop your added content if its too old by the time its needed. Unfortunately I'm not sure if the formats are documented clearly anywhere. Its about time they were so I will have a look around for you and look into writing some if necessary. HTH Amos That would be very helpful - I really appreciate it.
Re: [squid-users] Re: Re: Re: Re: Memory error with squid 3.3.4 and kerberos authentication
Now I can only access it on monday...
But looking at the code, i think this dp that's logged here is not
setted. This will be reached if a username has not a '@'.
if (!domain margs.ddomain) {
domain = xstrdup(margs.ddomain);
if (debug_enabled)
debug((char *) %s| %s: INFO: Got User: %s set default
domain: %s\n, LogTime(), PROGRAM, up, dp);
else
log((char *) %s| %s: INFO: Got User: %s set default
domain: %s\n, LogTime(), PROGRAM, up, dp);
}
On Fri, May 10, 2013 at 3:18 PM, Markus Moeller hua...@moeller.plus.com wrote:
Hi Carlos,
Could you run ext_kerberos_ldap_group_acl with -d and sent me the whole
output please ? It looks like a variable has not been freed and set to
NULL, but I can't see it in the code.
Thank you
Markus
Carlos Defoe carlosde...@gmail.com wrote in message
news:CAHsHsyvWT_AT4ZHFQ8Kv1QfPQeiKG1xzVLazZDYK=56g30=p...@mail.gmail.com...
Markus,
I tested a few times, and looks like the error occurs when one first
time i pass the testu...@my.domain.com to
ext_kerberos_ldap_group_acl, and then, following, i pass one simple
username, like testuser. That second time comes with the error, and
the Domain: part of the log comes with garbage. At the other error i
showed in previous message, it came empty, but look at this two other
errors:
testuser
kerberos_ldap_group.cc(429): pid=6430 :2013/05/10 12:31:42|
kerberos_ldap_group: INFO: Got User: testuser set default domain: 蒼
kerberos_ldap_group.cc(434): pid=6430 :2013/05/10 12:31:42|
kerberos_ldap_group: INFO: Got User: testuser Domain: 蒼
*** glibc detected ***
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk():
invalid pointer: 0x00611c90 ***
=== Backtrace: =
/lib64/libc.so.6[0x36a7475916]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9]
=== Memory map:
0040-00411000 r-xp fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
0061-00611000 rw-p 0001 fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
00611000-0164b000 rw-p 00:00 0 [heap]
31a420-31a4207000 r-xp fd:00 10375
/lib64/librt-2.12.so
31a4207000-31a4406000 ---p 7000 fd:00 10375
/lib64/librt-2.12.so
31a4406000-31a4407000 r--p 6000 fd:00 10375
/lib64/librt-2.12.so
testuser
kerberos_ldap_group.cc(429): pid=6361 :2013/05/10 12:30:05|
kerberos_ldap_group: INFO: Got User: testuser set default domain:
LOG_USER
kerberos_ldap_group.cc(434): pid=6361 :2013/05/10 12:30:05|
kerberos_ldap_group: INFO: Got User: testuser Domain: LOG_USER
*** glibc detected ***
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk():
invalid pointer: 0x00611ab0 ***
=== Backtrace: =
/lib64/libc.so.6[0x36a7475916]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9]
=== Memory map:
0040-00411000 r-xp fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
0061-00611000 rw-p 0001 fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
00611000-0164a000 rw-p 00:00 0 [heap]
31a420-31a4207000 r-xp fd:00 10375
/lib64/librt-2.12.so
31a4207000-31a4406000 ---p 7000 fd:00 10375
/lib64/librt-2.12.so
31a4406000-31a4407000 r--p 6000 fd:00 10375
/lib64/librt-2.12.so
In production, I assume the error appears when one user authenticate
with kerberos and then one another authenticate with basic.
On Thu, May 9, 2013 at 10:07 PM, Carlos Defoe carlosde...@gmail.com wrote:
Hello Markus,
That was from testing with gdb, so it was not properly basic
authentication, but a simple username (without kerberos nor netbios
domain), given to ext_kerberos_ldap_group_acl.
I will search in the cache logs for messages llike that, but i don't
think i was running the helper with debug options inside squid.
By the way, today i changed the auth to negotiate wrapper, now using
kerberos and ntlm, before the basic auth. If the memory problem is
from usernames received from basic authentication, i should see less
errors now. I also made one isolated clone of the proxy, for testing.
I will do more tests tomorrow.
On Thu, May 9, 2013 at 7:57 PM, Markus Moeller hua...@moeller.plus.com
wrote:
Hi Carlos,
The code will log INFO: Got User: %s set default domain: %s if the
user
has no domain and the default domain is used or INFO: Got User: %s
Domain:
NULL if the user has no domain and no default domain is provided.
If NTLM is used the code will log INFO: Got User: %s Netbios Name: %s
So I do not see a case where The following would be logged INFO: Got
User:
testusername Domain:.
[squid-users] Re: Re: Re: Re: Re: Memory error with squid 3.3.4 and kerberos authentication
Hi Carlos,
Yes you are right. I need to add
dp = xstrdup(rfc1738_escape(domain));
after
domain = xstrdup(margs.ddomain);
Markus
Carlos Defoe carlosde...@gmail.com wrote in message
news:cahshsyvuqa4a7uzkrotyd1i8t4cd3wwdruzzgzgr69_vfhx...@mail.gmail.com...
Now I can only access it on monday...
But looking at the code, i think this dp that's logged here is not
setted. This will be reached if a username has not a '@'.
if (!domain margs.ddomain) {
domain = xstrdup(margs.ddomain);
if (debug_enabled)
debug((char *) %s| %s: INFO: Got User: %s set default
domain: %s\n, LogTime(), PROGRAM, up, dp);
else
log((char *) %s| %s: INFO: Got User: %s set default
domain: %s\n, LogTime(), PROGRAM, up, dp);
}
On Fri, May 10, 2013 at 3:18 PM, Markus Moeller hua...@moeller.plus.com
wrote:
Hi Carlos,
Could you run ext_kerberos_ldap_group_acl with -d and sent me the whole
output please ? It looks like a variable has not been freed and set to
NULL, but I can't see it in the code.
Thank you
Markus
Carlos Defoe carlosde...@gmail.com wrote in message
news:CAHsHsyvWT_AT4ZHFQ8Kv1QfPQeiKG1xzVLazZDYK=56g30=p...@mail.gmail.com...
Markus,
I tested a few times, and looks like the error occurs when one first
time i pass the testu...@my.domain.com to
ext_kerberos_ldap_group_acl, and then, following, i pass one simple
username, like testuser. That second time comes with the error, and
the Domain: part of the log comes with garbage. At the other error i
showed in previous message, it came empty, but look at this two other
errors:
testuser
kerberos_ldap_group.cc(429): pid=6430 :2013/05/10 12:31:42|
kerberos_ldap_group: INFO: Got User: testuser set default domain: 蒼
kerberos_ldap_group.cc(434): pid=6430 :2013/05/10 12:31:42|
kerberos_ldap_group: INFO: Got User: testuser Domain: 蒼
*** glibc detected ***
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk():
invalid pointer: 0x00611c90 ***
=== Backtrace: =
/lib64/libc.so.6[0x36a7475916]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9]
=== Memory map:
0040-00411000 r-xp fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
0061-00611000 rw-p 0001 fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
00611000-0164b000 rw-p 00:00 0 [heap]
31a420-31a4207000 r-xp fd:00 10375
/lib64/librt-2.12.so
31a4207000-31a4406000 ---p 7000 fd:00 10375
/lib64/librt-2.12.so
31a4406000-31a4407000 r--p 6000 fd:00 10375
/lib64/librt-2.12.so
testuser
kerberos_ldap_group.cc(429): pid=6361 :2013/05/10 12:30:05|
kerberos_ldap_group: INFO: Got User: testuser set default domain:
LOG_USER
kerberos_ldap_group.cc(434): pid=6361 :2013/05/10 12:30:05|
kerberos_ldap_group: INFO: Got User: testuser Domain: LOG_USER
*** glibc detected ***
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl: munmap_chunk():
invalid pointer: 0x00611ab0 ***
=== Backtrace: =
/lib64/libc.so.6[0x36a7475916]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4031fa]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x36a741ecdd]
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl[0x4022b9]
=== Memory map:
0040-00411000 r-xp fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
0061-00611000 rw-p 0001 fd:00 150231
/usr/local/squid/libexec/ext_kerberos_ldap_group_acl
00611000-0164a000 rw-p 00:00 0 [heap]
31a420-31a4207000 r-xp fd:00 10375
/lib64/librt-2.12.so
31a4207000-31a4406000 ---p 7000 fd:00 10375
/lib64/librt-2.12.so
31a4406000-31a4407000 r--p 6000 fd:00 10375
/lib64/librt-2.12.so
In production, I assume the error appears when one user authenticate
with kerberos and then one another authenticate with basic.
On Thu, May 9, 2013 at 10:07 PM, Carlos Defoe carlosde...@gmail.com
wrote:
Hello Markus,
That was from testing with gdb, so it was not properly basic
authentication, but a simple username (without kerberos nor netbios
domain), given to ext_kerberos_ldap_group_acl.
I will search in the cache logs for messages llike that, but i don't
think i was running the helper with debug options inside squid.
By the way, today i changed the auth to negotiate wrapper, now using
kerberos and ntlm, before the basic auth. If the memory problem is
from usernames received from basic authentication, i should see less
errors now. I also made one isolated clone of the proxy, for testing.
I will do more tests tomorrow.
On Thu, May 9, 2013 at 7:57 PM, Markus Moeller hua...@moeller.plus.com
wrote:
Hi Carlos,
The code will log INFO: Got User: %s set default domain: %s if the
user
has no domain and the default domain is used or INFO: Got User: %s
Domain:
NULL if the user has no domain and no default domain is
[squid-users] SQUID / transparent proxying
I've been using SQUID for years to terminate inbound client connections to externally facing web sites. With SQUID 2.6, I specified transparent in the https_port, setup some acls, and it worked seamlessly. I have been trying to get a similar configuration working with SQUID 3.3. Changing the 'transparent' to intercept, adding ssl-bump, and then setting ssl_bump client-first to the appropriate domains. Unfortunately, I'm receiving these errors: 2013/05/10 18:33:11 kid1| NF getsockopt(SO_ORIGINAL_DST) failed on local=192.168.123.123:443 remote=4.4.4.4:11034 FD 12 flags=33: (92) Protocol not available Will this configuration still work with modern SQUID or must a different approach be taken? I appreciate any help, this is starting to frustrate me. Thanks, Warner CONFIDENTIALITY STATEMENT: All information included in this Communication, including attachment(s), is intended solely for delivery and authorized use by the intended recipient(s), and may contained privileged, confidential, proprietary and/or trade secret information entitled to protection and/or exempt from disclosure under applicable law. If you are not the intended recipient, please note that any use, distribution or copying of this Communication is unauthorized and may be unlawful. If you have received this Communication in error, please notify the sender and delete this Communication from your computer.
