On 5/7/06, Adam Back <[EMAIL PROTECTED]> wrote:
... Is this something one should be worrying about, or will tcp "take care of it" gracefully?
unless the application performs throttling in userspace (which is often poor; tricks like send/recv buffer sizing, nagle, etc can be used to improve accuracy) the TCP stack will try and optimize across all tcp sessions fairly. when many concurrent recv streams are active this implies not only latency and low bandwidth per stream but also likely invokes queuing at the DSL/cable ISP which can be as high as 2 seconds round trip. [this is a highly simplified example but common] overall your link is being well used, but per application / tcp session throughput is sluggish and constrained.
It seemed to me that at least for a while bittorrent suffered from this problem.
losing your ssh / vpn session in a flood of bittorrent neighbors is definitely painful without shaping. :)
Anyone have a tried and tested algorithm/method to address this?
traffic shaping in kernel / upstream before the ISP works best in my experience but does require a decent kernel (linux/bsd/*nix/cisco/*) for shaping at your host/router. if you need to rely on throttling at the application layer the use of small send buffers and disabled nagle gives you better consistency/accuracy when writing data at a regular interval on the socket. there are variations on this theme like LD_PRELOAD override of socket api and DiffServ/DSCP tagging which have their own particular strengths.
Generally what you'd like I think is QoS and to give the p2p app "bulk" precedence for other peoples traffic.
i suppose this is an open ended question (the ideal distribution of bandwidth among services on a link) but the following is my favorite configuration: - shape egress traffic such that buffering latency is avoided upstream to ISP - prioritize interactive traffic (ssh, not scp; vpn; rtp; chat; etc) - group bulk traffic into classes (p2p, backup, ftp, torrent, ?) utilizing spare capacity above an optional minimum throughput - leave large packets and other unclassified payload in a middle / unprioritized queue. i prefer iptables classifiers with the MARK target which is used by tc to configure the shaped queues and thresholds associated with each MARK. regarding user interaction, i'd love to see a learning mode applied to common/conservative traffic classification and shaping so that real world link capability as monitored over time was used to set more relevant thresholds / classification. this has the same limitations as explicit shaping but would be much more usable and let p2p / network application clients focus on features instead of poor traffic heuristics. you can find specific examples of "traffic shaping" for p2p in the archives and via google. citeseer has a huge number of papers on these subjects as well. _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
