Re: [squid-users] peek and splice content inspection question
Sorry to jump on a late thread - it is also possible to use ICAP/eCAP server to filter the actual contents of the stream. C-ICAP comes to mind first, then eCap samples from http://www.e-cap.org/Downloads And the *CAP services is a better solution than either URL-rewriting or chaining proxies. Since the HTTPS only gets MITM'd once, not twice or more. Amos ___ Hello All, I know DansGuardian, e2guardian, squidguard but no free solution with ICAP Do you have an advice for this ? Maybe there are some hubs, Squid icap DansGuardian ? William ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
Looking for one single thing that does everything DG or e2guardian do, or wraps them completely is the wrong approach. They are almost full-blown proxies like Squid. The *CAP design is to leave all the transfer proxying and caching duties to software like Squid and only perform the actual content adaptation policies in the service/module. You need to look at what DG is doing for you now and how much of that is available from squid.conf capabilities. Most of it usually is. Only the remaining fiddling with payloads bit actually needs a third-party service. Yes I know that Squid is very powerful, but (DG or E2) seems - to me - more easier with complex rules I'm working with multi users groups, regex in HTML, rules with exceptions (site allowed if some conditions), etc I guess if I reproduce my configuration in squid.conf it will be more hard to maintain after .. Also these soft are massively multi-threaded, in my usage squid + dg use less CPU than Squid alone I mean the load is shared by the cores. I also tried squid with SMP but there are some restrictions (delay pool, identification - if I remember right -) GreasySpoon or qlproxy seems to be the high profile ones. c-icap and Traffic Spicer seem to offer frameworks rather than pre-made filters http://www.squid-cache.org/Misc/icap.html The interfaces GreasySpoon or qlproxy describe say easily scriptable to do filtering. Though filtering is a huge topic, so easily is up for interpretation. Thanks, I will take a look And fiddling around with your customers content is very site-specific about what can or can't be done. Thus the frameworks and script engines being most high profile. Yes you are right, I wish that there is a change in E2, for me the ideal situation is SQUID (cache and identification) and a pool of E2 with ICAP I guess that there is no hope for SquidGuard because it's very different, a redirector related with Squid ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
On 23/08/2015 10:36 p.m., wmunny william wrote: Sorry to jump on a late thread - it is also possible to use ICAP/eCAP server to filter the actual contents of the stream. C-ICAP comes to mind first, then eCap samples from http://www.e-cap.org/Downloads And the *CAP services is a better solution than either URL-rewriting or chaining proxies. Since the HTTPS only gets MITM'd once, not twice or more. Amos ___ Hello All, I know DansGuardian, e2guardian, squidguard but no free solution with ICAP Do you have an advice for this ? Maybe there are some hubs, Squid icap DansGuardian ? Looking for one single thing that does everything DG or e2guardian do, or wraps them completely is the wrong approach. They are almost full-blown proxies like Squid. The *CAP design is to leave all the transfer proxying and caching duties to software like Squid and only perform the actual content adaptation policies in the service/module. You need to look at what DG is doing for you now and how much of that is available from squid.conf capabilities. Most of it usually is. Only the remaining fiddling with payloads bit actually needs a third-party service. GreasySpoon or qlproxy seems to be the high profile ones. c-icap and Traffic Spicer seem to offer frameworks rather than pre-made filters http://www.squid-cache.org/Misc/icap.html The interfaces GreasySpoon or qlproxy describe say easily scriptable to do filtering. Though filtering is a huge topic, so easily is up for interpretation. And fiddling around with your customers content is very site-specific about what can or can't be done. Thus the frameworks and script engines being most high profile. HTH Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
At least with squidguard you can't check the content (cookies, keywords in html, bad words, etc) It's just an URL/domains filter, but it can also block some objets contain in the request, eg http://foo.com/test.mp3, but it can't deny some kind of browsers or header informations. Maybe ufdbguard can, I don't know Sorry, I'm writing with my phone @Stanford a new release is coming ... ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
At least with squidguard you can't check the content (cookies, keywords in html, bad words, etc) It's just an URL/domains filter, but it can also block some objets contain in the request, eg http://foo.com/test.mp3, but it can't deny some kind of browsers or header informations. Maybe ufdbguard can, I don't know Sorry, I'm writing with my phone @Stanford a new release is coming ... ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
Hello Stanford and the list, Sorry to jump on a late thread - it is also possible to use ICAP/eCAP server to filter the actual contents of the stream. C-ICAP comes to mind first, then eCap samples from http://www.e-cap.org/Downloads Best regards, Rafael From: squid-users squid-users-boun...@lists.squid-cache.org on behalf of Stanford Prescott stan.presc...@gmail.com Sent: Monday, August 17, 2015 1:04 AM To: Yuri Voinov Cc: squid-users Subject: Re: [squid-users] peek and splice content inspection question Yes, really. ufdbGuard, like squidGuard before it, is a URL Filter that filters known unwanted URLs. A content filter, like DansGuardian and E2Guardian are content filters which examine the content of web pages looking for unwanted things. On Sun, Aug 16, 2015 at 6:10 PM, Yuri Voinov yvoi...@gmail.commailto:yvoi...@gmail.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 O, really? 17.08.15 4:03, Stanford Prescott ?: ufdbGuard is not a content filter. On Sun, Aug 16, 2015 at 4:07 PM, Yuri Voinov yvoi...@gmail.commailto:yvoi...@gmail.com wrote: ufdbguard does. 16.08.15 20:27, Stanford Prescott ?: I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.commailto:rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.orgmailto:squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJV0QpsAAoJENNXIZxhPexG24gIAMNuWsyfn/QkXWTXROZEJYL1 0frhC+w22fjV8svGjTrZEtSKY4LTHiHEjp99bPBEpPdoCURifUq20m018qRoIcEA XZfadD+s47bT7FvZbc2W58BQZUsWvotQRMNDPE+Mf0e38ev6PXsj16SaHmWytdx2 Z9H0y5qlgJwwbUyfps4uQn1wF16Qlf2Fw5TGRUbBrij+rjPYzDSXTXxtfT+4j/3V 4lZ0bN0HSFfvJrbfcpPoMCnSlRyJOm/b6Rxqv7v733OtrY/41EW1+HE1HOmW0em3 rwpAV1KgWrwMZYHcIBE147itXlz1RGQutX01auiiSvm/hO3h78rl6aSawmanOAM= =GgTR -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Either SquidGuard, or ufdbGuard has this functional onto blocking page. Just configure it. 17.08.15 5:04, Stanford Prescott пишет: Yes, really. ufdbGuard, like squidGuard before it, is a URL Filter that filters known unwanted URLs. A content filter, like DansGuardian and E2Guardian are content filters which examine the content of web pages looking for unwanted things. On Sun, Aug 16, 2015 at 6:10 PM, Yuri Voinov yvoi...@gmail.com wrote: O, really? 17.08.15 4:03, Stanford Prescott пишет: ufdbGuard is not a content filter. On Sun, Aug 16, 2015 at 4:07 PM, Yuri Voinov yvoi...@gmail.com yvoi...@gmail.com wrote: ufdbguard does. 16.08.15 20:27, Stanford Prescott пишет: I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJV0aHAAAoJENNXIZxhPexGQiMIAJAAW6E7JKROzwy0B+KcCLK6 BxfcA1o+lvJYwl6drsTeBH4NzO+Ra4eYmJMC94LwYc17E8Zwj5A+1t25cfQ1orIi 5EFjiVQ+0nseAGidcBnUNM1Nw+b4Xa/WswGo9+ApmslSstO1643uwteVip8o+Blg FuhYDuodynLNedsvxFq8/098zkZs1yc8d/pDyTAg4rQIGgU3gvxoMj3DLixFAkSy E0Qx0jEsSBvt0ksJAgxi0dVQh3ybeQqxevLgwDPFI0DuIvDh2Ho+6jwovzv+NyWS EK2i5CigTk2VguviWSFGmhpEkn3mvdLE5Kdj2XCB+1KFecmRv8ITwjvNNPKuh2w= =Mh/Z -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
On Sun, 16 Aug 2015 10:27:03 -0400 Stanford Prescott stan.presc...@gmail.com wrote: FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. Thanks for information about E2Guardian, I was not aware of it. Dansguardian didn't support kerberos authentication, and didn't have plans to ever support it, that's why I decided to move away from it to 'pure squid' setup. -- Marko Cupać https://www.mimar.rs/ ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
Yes, really. ufdbGuard, like squidGuard before it, is a URL Filter that filters known unwanted URLs. A content filter, like DansGuardian and E2Guardian are content filters which examine the content of web pages looking for unwanted things. On Sun, Aug 16, 2015 at 6:10 PM, Yuri Voinov yvoi...@gmail.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 O, really? 17.08.15 4:03, Stanford Prescott пишет: ufdbGuard is not a content filter. On Sun, Aug 16, 2015 at 4:07 PM, Yuri Voinov yvoi...@gmail.com yvoi...@gmail.com wrote: ufdbguard does. 16.08.15 20:27, Stanford Prescott пишет: I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJV0QpsAAoJENNXIZxhPexG24gIAMNuWsyfn/QkXWTXROZEJYL1 0frhC+w22fjV8svGjTrZEtSKY4LTHiHEjp99bPBEpPdoCURifUq20m018qRoIcEA XZfadD+s47bT7FvZbc2W58BQZUsWvotQRMNDPE+Mf0e38ev6PXsj16SaHmWytdx2 Z9H0y5qlgJwwbUyfps4uQn1wF16Qlf2Fw5TGRUbBrij+rjPYzDSXTXxtfT+4j/3V 4lZ0bN0HSFfvJrbfcpPoMCnSlRyJOm/b6Rxqv7v733OtrY/41EW1+HE1HOmW0em3 rwpAV1KgWrwMZYHcIBE147itXlz1RGQutX01auiiSvm/hO3h78rl6aSawmanOAM= =GgTR -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 O, really? 17.08.15 4:03, Stanford Prescott пишет: ufdbGuard is not a content filter. On Sun, Aug 16, 2015 at 4:07 PM, Yuri Voinov yvoi...@gmail.com wrote: ufdbguard does. 16.08.15 20:27, Stanford Prescott пишет: I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJV0QpsAAoJENNXIZxhPexG24gIAMNuWsyfn/QkXWTXROZEJYL1 0frhC+w22fjV8svGjTrZEtSKY4LTHiHEjp99bPBEpPdoCURifUq20m018qRoIcEA XZfadD+s47bT7FvZbc2W58BQZUsWvotQRMNDPE+Mf0e38ev6PXsj16SaHmWytdx2 Z9H0y5qlgJwwbUyfps4uQn1wF16Qlf2Fw5TGRUbBrij+rjPYzDSXTXxtfT+4j/3V 4lZ0bN0HSFfvJrbfcpPoMCnSlRyJOm/b6Rxqv7v733OtrY/41EW1+HE1HOmW0em3 rwpAV1KgWrwMZYHcIBE147itXlz1RGQutX01auiiSvm/hO3h78rl6aSawmanOAM= =GgTR -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 ufdbguard does. 16.08.15 20:27, Stanford Prescott пишет: I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJV0O2DAAoJENNXIZxhPexGd40H+wfKHdOmCXaxPQ9TJYvY/qQz mMLs2qLWae+TnC5Dt89SxOJ8oywodjyZo8BwZWeCnF+oYTjleUoglYxz2d9aJKNi rONKuAzsOXGekQ4GXv7t7CV2VtUljlppNccw7adWLAPnd4KwrbuRS73hHdoTV3eO 0V951eXnXOttVaW9IOL6kYh6jJtajZBkfoKoEMIR8d+q60vfM+tu7TiLAQHOjxPT SULJjB3U0swSlG70IwzD5HB/OwdDSlOCHB26A4BkTN9xoUn4gwxD2dRCxlNommtW MTF0A8s6lJexG4OQ9ci1E909HcfaE7iQJyFonj2st51m0P7aUC5r5DIs9A7/Fbc= =7hQ0 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
ufdbGuard is not a content filter. On Sun, Aug 16, 2015 at 4:07 PM, Yuri Voinov yvoi...@gmail.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 ufdbguard does. 16.08.15 20:27, Stanford Prescott пишет: I have SquidClamAV implemented with the Smoothwall Express 3.1 firewall. It works well and fast with ssl-bump, although the majority of our users only have relatively small networks with smaller loads. FYI, E2Guardian has replaced the DansGuardian project and is currently well maintained. E2Guardian can do content filtering for SSL but only in explicit mode, It currently does not support intercept (transparent) mode for SSLBump. On Fri, Aug 14, 2015 at 10:51 AM, Alex Rousskov rouss...@measurement-factory.com wrote: On 08/13/2015 10:31 PM, Amos Jeffries wrote: AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. The situation is more nuanced: eCAP supports asynchronous adapters. It is possible to write a ClamAV adapter that writes messages to disk and analyses them without blocking Squid. Doing so should eliminate most overheads between Squid and ClamAV. Factory ClamAV adapter can run in asynchronous mode, but threads are only used for _analysis_ of written files. We have not optimized the file writing part (yet?). Hopefully, using a RAM-based file system can mitigate a large part of that performance damage (as well as address some of the security concerns related to disk storage?). A bigger performance problem, AFAICT, is that ClamAV does not support incremental analysis. It waits for the entire file to be downloaded first. This breaks the message delivery pipeline and increases user-perceived response time. This problem cannot be solved outside the ClamAV library. Cheers, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJV0O2DAAoJENNXIZxhPexGd40H+wfKHdOmCXaxPQ9TJYvY/qQz mMLs2qLWae+TnC5Dt89SxOJ8oywodjyZo8BwZWeCnF+oYTjleUoglYxz2d9aJKNi rONKuAzsOXGekQ4GXv7t7CV2VtUljlppNccw7adWLAPnd4KwrbuRS73hHdoTV3eO 0V951eXnXOttVaW9IOL6kYh6jJtajZBkfoKoEMIR8d+q60vfM+tu7TiLAQHOjxPT SULJjB3U0swSlG70IwzD5HB/OwdDSlOCHB26A4BkTN9xoUn4gwxD2dRCxlNommtW MTF0A8s6lJexG4OQ9ci1E909HcfaE7iQJyFonj2st51m0P7aUC5r5DIs9A7/Fbc= =7hQ0 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
On Fri, 14 Aug 2015 03:38:47 +1200 Amos Jeffries squ...@treenet.co.nz wrote: On 14/08/2015 12:47 a.m., Marko Cupać wrote: Hi, a few years ago I had a working setup of squid + dansguardian which was giving me ability to inspect traffic and filter it according to various criteria, mainly extensions, mime types and presence of malicious code (clamav). Lately most of the web moved to https, and dansguardian isn't maintained for almost three years, which made my setup obsolete. Is it possible - by means of squid's peek and splice feature - to inspect file extensions and mime types of https traffic? Can bumped https traffic be forwarded to icap (squidclamav) for AV scanning? Doing so is the features intended purpose. And finally, would overly curious and unethical admin be able to easily dump bumped data and find sensitive information there? When correctly used TLS cannot be decrypted. BUt, most use of HTTPS today is not using TLS correctly. If it could be bumped at all then it could be dumped as easily as inspected by an AV. Like a sharp knife can be as easily used for cutting vegetables as throats. Ones intent has nothing to do with the tools capability or lack. I completely agree with you, I shouldn't have mixed intent with capability which is great and which I intend to put to good use. So, if I understand well, if I just send traffic to squidclamav on icap tcp port, then I don't store usernames and passwords or private emails in cache? This is important to me in order to explain the complete mechanism to management and to create understandable policy for end users. -- Marko Cupać https://www.mimar.rs/ ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 14.08.15 2:02, Marko Cupać пишет: On Fri, 14 Aug 2015 03:38:47 +1200 Amos Jeffries squ...@treenet.co.nz wrote: On 14/08/2015 12:47 a.m., Marko Cupać wrote: Hi, a few years ago I had a working setup of squid + dansguardian which was giving me ability to inspect traffic and filter it according to various criteria, mainly extensions, mime types and presence of malicious code (clamav). Lately most of the web moved to https, and dansguardian isn't maintained for almost three years, which made my setup obsolete. Is it possible - by means of squid's peek and splice feature - to inspect file extensions and mime types of https traffic? Can bumped https traffic be forwarded to icap (squidclamav) for AV scanning? Doing so is the features intended purpose. And finally, would overly curious and unethical admin be able to easily dump bumped data and find sensitive information there? When correctly used TLS cannot be decrypted. BUt, most use of HTTPS today is not using TLS correctly. If it could be bumped at all then it could be dumped as easily as inspected by an AV. Like a sharp knife can be as easily used for cutting vegetables as throats. Ones intent has nothing to do with the tools capability or lack. I completely agree with you, I shouldn't have mixed intent with capability which is great and which I intend to put to good use. So, if I understand well, if I just send traffic to squidclamav on icap tcp port, then I don't store usernames and passwords or private emails in cache? I would not worry about it. No physical access to the cache such data does not pull out with proper administration. Unless, of course, do not put a proxy in a phone booth on the street. If it starts to bother me - I either start using encrypted file system, or build a completely black box - completely disable logging of user access. This is important to me in order to explain the complete mechanism to management and to create understandable policy for end users. -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJVzQP8AAoJENNXIZxhPexGNDUIAMhXUmakjPIpSBlEcb2CsEZN gS3b6iTLKo2YnBqr2NU1TV9/fqrDZIqd/lszlIta5phYmkiKcRGLP4bR87+SW7ze dBGeAZeDehXWv4Ga7/YlmAB6LpWRC3Yd0lm3WTiZ/AnowcaxOHx/Q/H7DhDiIFEN HRDjRGoTcoIkNP+BC76AnrF+8MErz0cPMXLBqVCXNR+ijNCP9LBza1Y5h88QqX7U cpRaj88LsW7pQeNHNMDtO7PneNKzho/YUO+M0BTtHXw4Mdwdqt1MBViXhTTh/GP9 C5A1DDLvr384YmoG0eReEt/KVIBliTV80htmn6lYT5dJiX2Fu+TAOEjohz+nkcc= =T28k -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 14.08.15 2:56, Alex Rousskov пишет: On 08/13/2015 09:38 AM, Amos Jeffries wrote: On 14/08/2015 12:47 a.m., Marko Cupać wrote: Is it possible - by means of squid's peek and splice feature - to inspect file extensions and mime types of https traffic? Can bumped https traffic be forwarded to icap (squidclamav) for AV scanning? Doing so is the features intended purpose. And you may be able to use either Secure ICAP (Squid 4) or the eCAP ClamAV adapter for AV scanning without transmitting bumped messages over plain text ICAP connections. Yet another solution is not transmit any over net. Just setup all services on blade system or one box. if I just send traffic to squidclamav on icap tcp port, then I don't store usernames and passwords or private emails in cache? Squid caching is not related to AV scanning. If you do not disable caching, Squid will cache cachable responses. IIRC, the code making the cachability decision does not check whether the response was bumped. However, you may configure it to do so using the cache directive: http://www.squid-cache.org/Doc/config/cache/ Said that, most responses with private information should not be cachable by default because the server should mark them as such. ... and we ignore them due to abuse of server owners no-cache directives when we fight for increase hit-ratio. There is millions cache-unfriendly web servers, starting from Google... The current eCAP ClamAV adapter [temporary] stores message bodies on disk to pass them to the ClamAV library for analysis. I do not know about squidclamav. HTH, Alex. ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBCAAGBQJVzQkBAAoJENNXIZxhPexGGDMH/jkrsMwvbDgWADFxcrapPZl8 XCi0fcJTGhO1GPvhBB/T505HMDiwoCeMU5A329i3CWpMXEPJqkllJ0AtPYrcwp7l gL21HOx50Cqv8rWL4bZR7k9wfb3smLN/NNBZSN6HXZh1chkRhlal+x5qXcfvB7BY +uJIRnVet0uCQoAHdXuUBH0Qlo+tVaFtlywBRdwNO84uDgW8VaKB4sruV8YO3/Em wS55QU8nCezaIYaP014LRjh6vpAQfcer5i4FqapGMVe0Qt3VY752ayBl0hN0REN1 kdGoLgvY0263WlWvdbdGB4W1oearfKZzDXUjvwmcTiY0WzpeV+B/XlYFze3w+pg= =f521 -END PGP SIGNATURE- ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
Re: [squid-users] peek and splice content inspection question
On 14/08/2015 9:15 a.m., Yuri Voinov wrote: 14.08.15 2:56, Alex Rousskov пишет: On 08/13/2015 09:38 AM, Amos Jeffries wrote: On 14/08/2015 12:47 a.m., Marko Cupać wrote: Is it possible - by means of squid's peek and splice feature - to inspect file extensions and mime types of https traffic? Can bumped https traffic be forwarded to icap (squidclamav) for AV scanning? Doing so is the features intended purpose. And you may be able to use either Secure ICAP (Squid 4) or the eCAP ClamAV adapter for AV scanning without transmitting bumped messages over plain text ICAP connections. Yet another solution is not transmit any over net. Just setup all services on blade system or one box. Like Alex said the design of Clam' AV and toolchain is that it uses disk storage for relaying objects between processes. There are some popular security policies where disk storage is forbidden. if I just send traffic to squidclamav on icap tcp port, then I don't store usernames and passwords or private emails in cache? Squid caching is not related to AV scanning. If you do not disable caching, Squid will cache cachable responses. IIRC, the code making the cachability decision does not check whether the response was bumped. However, you may configure it to do so using the cache directive: http://www.squid-cache.org/Doc/config/cache/ Or alternatively use a memory-only proxy cache. This allows a large portion of the caching HIT benefits to still be gained without violating any security requirements about persistent storage of TLS or HTTPS message data. That only covers the Squid cache storage part of the system though. Said that, most responses with private information should not be cachable by default because the server should mark them as such. ... and we ignore them due to abuse of server owners no-cache directives when we fight for increase hit-ratio. There is millions cache-unfriendly web servers, starting from Google... No Yuri. The confusing no-cache control fequently used means only that the proxy needs to revalidate the cache HIT content and headers before delivering to the client. All current Squid releases do that correctly. The squid.conf settings once available to ignore/override are no longer existing. Alex was talking of private and no-store directives. Their meaning is clear and precise - not easily confused. Overriding those is somewhat stupid. The current eCAP ClamAV adapter [temporary] stores message bodies on disk to pass them to the ClamAV library for analysis. I do not know about squidclamav. It seemed to do the same when I checked it a few months ago. AFAICS it is the backend AV library only scanning disk objects that causes the whole issue. Otherwise the eCAP could be much, much faster. Amos ___ squid-users mailing list squid-users@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-users
