Re: [squid-users] ssl_bump for specific dstdomain

[Daniel Greenwald]

here is a python helper I wrote with help of previous posts. It takes
sni info from squid and returns OK if the domain is in
/etc/squid/domains_nobump.acl (I am not a coder..) Problem is it works
good for intercepted browser traffic but doesn't work when a user
tries to use an app on an eg android device. In my cache.log I get:
 error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca

It seems Squid doesn't send SNI info before the error shows up in
cache.log and the app barfs. I'm guessing the app is detecting the
SSLBump before squid is able to grab the SNI. Does this makes sense?
Any ideas to move this forward?


#!/usr/bin/python
import sys
import string
lines = [line.strip() for line in open('/etc/squid/domains_nobump.acl')]

while True:
req = sys.stdin.readline()
req = req.strip()
if not req:
break

try:
id, sni = req.split()
sys.stderr.write('request %r\n' % req)
sys.stderr.flush()
for line in lines:
if line.startswith('.'):
if string.find(sni,line,len(sni)-len(line)) != -1 or
sni == line.lstrip('.'):  # bypass
sys.stdout.write('{} OK\n'.format(id))
sys.stdout.flush()
break
else:
if sni == line:
sys.stdout.write('{} OK\n'.format(id))
sys.stdout.flush()
break

else:
sys.stdout.write('{} ERR\n'.format(id))
sys.stdout.flush()
except:
sys.stderr.write('SNICHECK INPUT: %r\n' % req)


squid.conf:

external_acl_type sni ttl=30 concurrency=60 children-max=3
children-startup=1 %ssl::>sni /usr/lib64/squid/snicheck.py
acl sni_nobump external sni
ssl_bump splice sni_nobump
ssl_bump peek step1 all
ssl_bump bump step2 all
---
Daniel I Greenwald




On Fri, Mar 13, 2015 at 5:04 AM, Amos Jeffries  wrote:
> On 13/03/2015 6:39 p.m., Yuri Voinov wrote:
>>
>>
>> 13.03.15 2:37, Mukul Gandhi пишет:
>>> On Thu, Mar 12, 2015 at 11:04 AM, Yuri Voinov 
>>> wrote:
>>
>>> You only have external helper (which is must wrote yourself) in
>>> 3.4.x.
>>
>>
 Are there any examples that I can look at to implemented this
 external helper for doing selective ssl_bumps. And what would
 this helper script do anyways? All we have is the destination IP
 address which is not really going to give us the actual HTTP
 hostname.
>> Yes and no. There is one third-party helper in list archives, written
>> on python. No one of this including in squid distribution.
>>
>>
>>> Works with domains in ssl bump fully available at least 3.5.x
>>
>>
 Does the 3.5.x implementation decrypt the whole payload and then
 do the ssl_bump? The "peek" option seems to imply that only the
 HTTP headers are peeked at.
>> Of course. As by 3.4.x. The difference is only with mechanisms.
>
> And no at the same time. HTTP message headers inside the encryption are
> encrypted and unavailable until after the decryption is decided (bumped).
>
> What gets peeked at is the TLS ClientHello and TLS ServerHello details.
> SNI may become available by peeking when raw-IP was all that was in the
> HTTP CONNECT message or intercepted TCP packets.
>
> You can then use those non-private TLS details to decide between reject,
> splice (pass-thru) or bump (decrypt) for the encrypted HTTPS data.
>
>
>>
 I guess what I am asking is, is there any way we can do this
 without actually decrypting the payload?
>> 3.5.x peek-and-splise functionality do bump splitted by stages.
>> Against 3.4.x, which is makes bump in one stage.
>>
>
> Amos
>
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

[squid-users] mmap() in squid

[Eugene M. Zheganin]

Hi.

Squid uses mmap() call from 3.4.x, and mmap() on FreeBSD it has one
specific flag - MAP_NOSYNC, which prevents dirtied pages from being
flushed on disk:

MAP_NOSYNCCauses data dirtied via this VM map to be flushed to
   physical media only when necessary (usually by the
   pager) rather than gratuitously.  Typically this pre-
   vents the update daemons from flushing pages dirtied
   through such maps and thus allows efficient
sharing of
   memory across unassociated processes using a file-
   backed shared memory map.  Without this option any VM
   pages you dirty may be flushed to disk every so often
   (every 30-60 seconds usually) which can create
perfor-
   mance problems if you do not need that to occur (such
   as when you are using shared file-backed mmap regions
   for IPC purposes).  Note that VM/file system
coherency
   is maintained whether you use MAP_NOSYNC or not. 
This
   option is not portable across UNIX platforms (yet),
   though some may implement the same behavior by
default.

   WARNING!  Extending a file with ftruncate(2),
thus cre-
   ating a big hole, and then filling the hole by
modify-
   ing a shared mmap() can lead to severe file
fragmenta-
   tion.  In order to avoid such fragmentation you
should
   always pre-allocate the file's backing store by
   write()ing zero's into the newly extended area
prior to
   modifying the area via your mmap().  The
fragmentation
   problem is especially sensitive to MAP_NOSYNC pages,
   because pages may be flushed to disk in a totally
ran-
   dom order.

   The same applies when using MAP_NOSYNC to implement a
   file-based shared memory store.  It is
recommended that
   you create the backing store by write()ing zero's to
   the backing file rather than ftruncate()ing it.  You
   can test file fragmentation by observing the KB/t
   (kilobytes per transfer) results from an ``iostat 1''
   while reading a large file sequentially, e.g. using
   ``dd if=filename of=/dev/null bs=32k''.

   The fsync(2) system call will flush all dirty
data and
   metadata associated with a file, including dirty
NOSYNC
   VM data, to physical media.  The sync(8) command and
   sync(2) system call generally do not flush dirty
NOSYNC
   VM data.  The msync(2) system call is obsolete since
   BSD implements a coherent file system buffer cache.
   However, it may be used to associate dirty VM pages
   with file system buffers and thus cause them to be
   flushed to physical media sooner rather than later.

Last year there was an issue with PostgreSQL, which laso started to use
mmap() in it's 9.3 release, and it had a huge regression issue on
FreeBSD. One of the measures to fight this regression (but not the only)
was adding MAP_NOSYNC to the postgresql port. So I decided to do the
same for my local squid. I created a patch, where both of two
occurencies of mmap() were supplied with this flag. I'm using squid
3.4.x patched this way about a half-a-year. Couple of days ago I sent
this patch to the FreeBSD ports system, and squid port maintainer asks
me if I'm sure squid on FreeBSD does need this. Since I'm not a skilled
programmer (though I think using mmap() with MAP_NOSYNC is a good
thing), I decided to ask here - is this flag worth bothering, since
squid isn't a database engine ?

Thanks.
___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

Re: [squid-users] assertion failed: ../src/ipc/AtomicWord.h:88: "Enabled()"

[Amos Jeffries]

Hi Dan,
 This appears by a breakage in the 3.5 snapshots' GNU atomics detection.
Though we are still not sure why the error occurs yet with atomics disabled.

Snapshots labelled r13783 or later available in a few hrs should be fixed.

Cheers
Amos


On 27/03/2015 11:47 a.m., Dan Charlesworth wrote:
> Bumping this because I think it might have gone into the black hole the other 
> night.
> 
>> On 23 Mar 2015, at 5:44 pm, Dan Charlesworth  wrote:
>>
>> Turns out it’s also shitting the bed whenever I go to an SSL site now that 
>> I’ve added --enable-storeio=rock:
>>
>> 2015/03/23 17:40:13 kid1| assertion failed: ../src/ipc/AtomicWord.h:71: 
>> "Enabled()"
>> 2015/03/23 17:42:02 kid1| assertion failed: ../src/ipc/AtomicWord.h:74: 
>> "Enabled()"
>>
>> I feel like I’m definitely missing a dependency or something :-/
>>
>>> On 23 Mar 2015, at 5:28 pm, Dan Charlesworth >> > wrote:
>>>
>>> Hey!
>>>
>>> Sorry for all the threads lately, folks -
>>>
>>> I just recompiled by 3.5 EL6 (64-bit) RPM (using 
>>> squid-3.5.2-20150321-r13782).
>>>
>>> I decided to add rock to my `—enable-storeio` option, so I could try SMP 
>>> and stuff, which was fine. But when I went to squid -z it, I got this crash:
>>> assertion failed: ../src/ipc/AtomicWord.h:88: "Enabled()"
>>>
>>> Just using:
>>> cache_dir rock /var/spool/squid 2
>>> workers 2
>>>
>>> I’m hoping, for a change, this is some obvious thing I’ve missed and not 
>>> something I need to dig out backtraces for :-)
>>>
>>> Thanks, y'all
>>
> 
> 
> 
> 
> ___
> squid-users mailing list
> squid-users@lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 

___
squid-users mailing list
squid-users@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users