Re: [LARTC] How to fight with encrypted p2p
About ipp2p, Right now, the battle against p2p is lost with l7 detection from ipp2p, l7 filter and others. Why ?? It is a known fact that pattern matching does not work with full encrypted P2P handshakes based on DHT key exchange algorithms with byte padding. You have absolutely no byte pattern and no fixed packet lengths in the stream. So something like a flow history will fail or might have a very high false +ve rate. The thing is that there are proprietary solutions which can detect fully encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is selling a proprietary library for this which is integrated in some firewall vendors). I have not seen any open source development into this direction. Klaus, (former) maintainer of ipp2p Marco Aurelio wrote: As you might have seen, these are words from ipp2p author: I have seen some pieces of code from ipoque which can detect encypted bittorrent and edonkey traffic. Unforunately, this code will not work with iptables, because it needs more information about the flow history and the history of an ip address. Right now, I do not have the time and the money to develop a filter like this, but if you are interested in a developement in this direction, please contact me. I *think* that we need something like a bittorrent helper in the kernel to keep this extra information about the flow history and then an iptables plugin to match. What do you think? Maybe we could contact him to know what kind of information is it? On Nov 12, 2007 9:17 AM, sawar [EMAIL PROTECTED] wrote: Rtorrent which I use sometimes have ability to completely disable plain text communication : man rtorrent allow_incoming (allow incoming encrypted connections), try_outgoing (use encryption for outgoing connections), require (disable unencrypted handshakes), require_RC4 (also disable plaintext transmission after the initial encrypted handshake), enable_retry (if the initial outgoing connection fails, retry with encryption turned on if it was off or off if it was on), prefer_plain text (choose plaintext when peer offers a choice between plaintext transmission and RC4 encryption, otherwise RC4 will be used). and many other clients have similar abilities. I'm afraid that full encrypted and enabled by default communication is only a matter of time and we will lose this fight very soon. Some clients P2P clients are nice about there encryption and negotiate encryption ahead of time using plain communication. I.E. Limewire, Azureus. However, some just start TLS and that is all you can see. Looking at ipp2ps signatures, I don't see anything that leads me to believe they track that kind of info. David Bierce On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote: sAwAr wrote: Hi I believe that whole question is in topic. Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? Modern p2p clients have this ability moreover some of them have this enabled by default. Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic. Thanks in advance. Pozdrawiam Szymon Turkiewicz Have not tried this. An idea. P2P initiations are not encrypted AFAIK. Thus connections can be marked and related traffic shaped. If initiation is also encrypted, then I think we have a serious problem. Mohan ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] How to fight with encrypted p2p
Sorry ... I'm little bite tired ... I mean that we might sponsor Klauss and L7 team to develop this ... Regards Sébastien CRAMATTE escribió: Klauss, Could you Might be you can sponsor the development ... Regards Sébastien Klaus escribió: About ipp2p, Right now, the battle against p2p is lost with l7 detection from ipp2p, l7 filter and others. Why ?? It is a known fact that pattern matching does not work with full encrypted P2P handshakes based on DHT key exchange algorithms with byte padding. You have absolutely no byte pattern and no fixed packet lengths in the stream. So something like a flow history will fail or might have a very high false +ve rate. The thing is that there are proprietary solutions which can detect fully encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is selling a proprietary library for this which is integrated in some firewall vendors). I have not seen any open source development into this direction. Klaus, (former) maintainer of ipp2p Marco Aurelio wrote: As you might have seen, these are words from ipp2p author: I have seen some pieces of code from ipoque which can detect encypted bittorrent and edonkey traffic. Unforunately, this code will not work with iptables, because it needs more information about the flow history and the history of an ip address. Right now, I do not have the time and the money to develop a filter like this, but if you are interested in a developement in this direction, please contact me. I *think* that we need something like a bittorrent helper in the kernel to keep this extra information about the flow history and then an iptables plugin to match. What do you think? Maybe we could contact him to know what kind of information is it? On Nov 12, 2007 9:17 AM, sawar [EMAIL PROTECTED] wrote: Rtorrent which I use sometimes have ability to completely disable plain text communication : man rtorrent allow_incoming (allow incoming encrypted connections), try_outgoing (use encryption for outgoing connections), require (disable unencrypted handshakes), require_RC4 (also disable plaintext transmission after the initial encrypted handshake), enable_retry (if the initial outgoing connection fails, retry with encryption turned on if it was off or off if it was on), prefer_plain text (choose plaintext when peer offers a choice between plaintext transmission and RC4 encryption, otherwise RC4 will be used). and many other clients have similar abilities. I'm afraid that full encrypted and enabled by default communication is only a matter of time and we will lose this fight very soon. Some clients P2P clients are nice about there encryption and negotiate encryption ahead of time using plain communication. I.E. Limewire, Azureus. However, some just start TLS and that is all you can see. Looking at ipp2ps signatures, I don't see anything that leads me to believe they track that kind of info. David Bierce On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote: sAwAr wrote: Hi I believe that whole question is in topic. Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? Modern p2p clients have this ability moreover some of them have this enabled by default. Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic. Thanks in advance. Pozdrawiam Szymon Turkiewicz Have not tried this. An idea. P2P initiations are not encrypted AFAIK. Thus connections can be marked and related traffic shaped. If initiation is also encrypted, then I think we have a serious problem. Mohan ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] ip_conntrack: falling back to vmalloc.
Hello I've got a server with 3Gb of ram and I want to keep 256 for the system and allocate the rest to conntrack ... I've tried to change the HASHSIZE of the ip_conntrack but dmesg return me this error ! ip_conntrack version 2.4 (2097152 buckets, 16777216 max) - 236 bytes per conntrack ip_conntrack: falling back to vmalloc. I've use this math to calculate it : (3072 - 256) x 1024^2 - 236 = 12511822,1027 The near power of 2 seems to be 2^23 = 8388608 With this result I've change my sysctl.conf file net.ipv4.netfilter.ip_conntrack_max = 8388608 net.ipv4.netfilter.ip_conntrack_tcp_timeout_established= 28800 and I've to change the HASHSIZE to ip_conntrack_max / 4 ... What is wrong ! How can I solve the problem ... I'm waiting for a server with 8Gb (8192) of ram most of available to use with conntrack ! Regards ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Hardware Requirements for qdisc htb/sfq
I am planning to replace our cisco 7200 core router with Linux. We currently serve around 1500 (3/4 DSL - different router) customers with probably half of them being concurrent at any given time. We have a fiber network and customers currently aren't managed as far as how much bandwidth they can use at anytime. Therefore I have constructed a working tc qdisc Linux router as a test. It is working beautifully. My question is what are the general hardware requirements for routing to about 20 subnets (class c), traffic shaping for about 50 fiber customers (TC QDISC), 2 T1s (straight into the Linux router) and about 35MB of traffic out to the next ISP? We are planning to implement BGP sometime in the near future. I have been searching everywhere for some kind of guidelines, but I see none. If anyone could give me a round about answer that would point me in the right direction I would be obliged. This is what I have been looking at: 2.0GHz Dual-Core Xeon, 4GB 667MGz RAM, 2x1Gbit Network Interfaces. Is this overkill? Thanks, Shane McKinley ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] Hardware Requirements for qdisc htb/sfq
This is what I am getting atm: 5 minute input rate 21323000 bits/sec, 3544 packets/sec 5 minute output rate 787 bits/sec, 3084 packets/sec So I should prolly be good with the hardware listed, huh? I am mostly concerned about the qdisc stuff, is it more CPU intensive or RAM? -Original Message- From: Marek Kierdelewicz [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 14, 2007 4:36 PM To: lartc@mailman.ds9a.nl Subject: Re: [LARTC] Hardware Requirements for qdisc htb/sfq My question is what are the general hardware requirements for routing to about 20 subnets (class c), traffic shaping for about 50 fiber customers (TC QDISC), 2 T1s (straight into the Linux router) and about 35MB of traffic out to the next ISP? We are planning to implement BGP sometime in the near future. From my experience router load is mostly dependant on pps (packet per second). I was doing statefull 550k pps on Athlon64 X2 5200 cpu usage was ~50% per core. I'm doing stateless 1m pps on Quad-core Xeon(R) CPU E5345 @ 2.33GHz, peak cpu usage is less then 25% per core both configs used e1000 nics, 1GB Ram both boxes were running BGP I have been searching everywhere for some kind of guidelines, but I see none. If anyone could give me a round about answer that would point me in the right direction I would be obliged. Use irqbalance of smp affinity to distribute irqs between cores. Make your router stateless or optimeze netfilter settings related to conntrack (more memory, bigger hashtables ect) 2.0GHz Dual-Core Xeon, 4GB 667MGz RAM, 2x1Gbit Network Interfaces. Quite a lot of ram for statefull firewall + BGP (1GB would probably suffice). As for CPU diagnose your network and compare your pps with the numbers and platforms I specified. cheers, Marek Kierdelewicz KoBa ISP ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Hardware Requirements for qdisc htb/sfq
So I should prolly be good with the hardware listed, huh? I am mostly concerned about the qdisc stuff, is it more CPU intensive or RAM? Shaping is not RAM hogging at all. With simpler setups It should not be too CPU intensive either. If you plan to do per-user shaping then consider using hashing u32 filters. cheers, Marek Kierdelewicz KoBa ISP ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Hardware Requirements for qdisc htb/sfq
My question is what are the general hardware requirements for routing to about 20 subnets (class c), traffic shaping for about 50 fiber customers (TC QDISC), 2 T1s (straight into the Linux router) and about 35MB of traffic out to the next ISP? We are planning to implement BGP sometime in the near future. From my experience router load is mostly dependant on pps (packet per second). I was doing statefull 550k pps on Athlon64 X2 5200 cpu usage was ~50% per core. I'm doing stateless 1m pps on Quad-core Xeon(R) CPU E5345 @ 2.33GHz, peak cpu usage is less then 25% per core both configs used e1000 nics, 1GB Ram both boxes were running BGP I have been searching everywhere for some kind of guidelines, but I see none. If anyone could give me a round about answer that would point me in the right direction I would be obliged. Use irqbalance of smp affinity to distribute irqs between cores. Make your router stateless or optimeze netfilter settings related to conntrack (more memory, bigger hashtables ect) 2.0GHz Dual-Core Xeon, 4GB 667MGz RAM, 2x1Gbit Network Interfaces. Quite a lot of ram for statefull firewall + BGP (1GB would probably suffice). As for CPU diagnose your network and compare your pps with the numbers and platforms I specified. cheers, Marek Kierdelewicz KoBa ISP ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Hardware Requirements for qdisc htb/sfq
Shane McKinley wrote: I have been searching everywhere for some kind of guidelines, but I see none. If anyone could give me a round about answer that would point me in the right direction I would be obliged. This is what I have been looking at: 2.0GHz Dual-Core Xeon, 4GB 667MGz RAM, 2x1Gbit Network Interfaces. Is this overkill? Speed normally seen in PPS. The 7200 routes approx 1M PPS. I ran our own routing and classification s/w on a AMD Opteron 2Ghz, 1GB RAM and got 1.1M PPS. Linux Kernel gave approx 700K PPS. IMHO your h/w is way oversized but why not? H/w is cheap nowadays. Mohan ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
