Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
> > Well, this is all absolutely sad. Now we (nearly) have new kqueue, it > > does fly, but I can't break above 32k anyway .. > > Epoll on Linux? what is linux? ;) with the monkey patch to use new IO, I can use the Little Daemon flying on kqueue .. so problem solved. __

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Itamar Turner-Trauring
> Well, this is all absolutely sad. Now we (nearly) have new kqueue, > it does fly, but I can't break above 32k anyway .. Epoll on Linux? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo

Re: [Twisted-Python] Using Trial with Eclipse and PyDev

2011-11-14 Thread Glyph
On Nov 14, 2011, at 10:39 AM, Tom Sheffler wrote: > I've seen a few answers regarding this topic around the net, but nothing > recent. > > I've enjoyed using the Twisted code-base in Eclipse, and have also liked > running unittests in the PyDev framework. However, I haven't been successful >

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Glyph
On Nov 14, 2011, at 1:48 PM, Corbin Simpson wrote: > ... I think it's a lot better than letting us rely on Github. I don't want to "rely" on github. I want our official presence there (which is already there) to be current and properly maintained so that people who use Github for other reason

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Corbin Simpson
On Mon, Nov 14, 2011 at 10:44 AM, Corbin Simpson wrote: > On Mon, Nov 14, 2011 at 10:33 AM, Glyph wrote: >> On Nov 14, 2011, at 6:55 AM, Tim Allen wrote: >> >> It's very well to say "make git mirror, push to Github, get new >> contributors", but I think there's a social impedance mismatch here >>

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
> > I was told that the new Python 3 IO system does not use fopen(), > > however Twisted is not yet there on Python 3, right? > > If you are wild, you can try https://bitbucket.org/pitrou/t3k/ (at least as an > experiment :-)) I am .. but not thaaat much;) > > > There is a backport of that new

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Corbin Simpson
On Mon, Nov 14, 2011 at 10:33 AM, Glyph wrote: > On Nov 14, 2011, at 6:55 AM, Tim Allen wrote: > > It's very well to say "make git mirror, push to Github, get new > contributors", but I think there's a social impedance mismatch here > that's going to cause problems, or at least make people wary be

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Glyph
On Nov 14, 2011, at 6:55 AM, Tim Allen wrote: > It's very well to say "make git mirror, push to Github, get new > contributors", but I think there's a social impedance mismatch here > that's going to cause problems, or at least make people wary because > Twisted's Github project behaves weirdly an

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Antoine Pitrou
On Mon, 14 Nov 2011 09:57:46 -0800 Tobias Oberstein wrote: > Now I'm running out of options. > > I was told that the new Python 3 IO system does not use fopen(), > however Twisted is not yet there on Python 3, right? If you are wild, you can try https://bitbucket.org/pitrou/t3k/ (at least as an

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
> That's definitely not a limit on Python or Twisted. That's an OS limit. > Are you opening lots of files in addition to your sockets? As it turns out, you are right, and the full answer is most unpleasant: This http://www.freebsd.org/cgi/query-pr.cgi?pr=148581&cat= bug in FreeBSD libc, which i

[Twisted-Python] Using Trial with Eclipse and PyDev

2011-11-14 Thread Tom Sheffler
I've seen a few answers regarding this topic around the net, but nothing recent. I've enjoyed using the Twisted code-base in Eclipse, and have also liked running unittests in the PyDev framework. However, I haven't been successful integrating Trial tests with the PyDev runner. I did get Trial to

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
> What does resource.getrlimit(resource.RLIMIT_OFILE) return? > > The Python docs say: > > resource.RLIMIT_OFILE > The BSD name for RLIMIT_NOFILE. It's not there .. neither 2.6 nor 2.7 [autobahn@autobahnhub ~/AutobahnHub/service]$ python Python 2.7.1 (r271:86832, Dec 13 2010, 15:52:15) [GCC 4.

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Donal McMullan
What does resource.getrlimit(resource.RLIMIT_OFILE) return? The Python docs say: resource.RLIMIT_OFILE The BSD name for RLIMIT_NOFILE. D On 15 November 2011 03:57, Tobias Oberstein wrote: >> > 2) Too many files. >> > Is there another limit specifically for files, and/or something tunable in

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
> > 2) Too many files. > > Is there another limit specifically for files, and/or something tunable in > Python/Twisted? > > That's definitely not a limit on Python or Twisted. That's an OS limit. > Are you opening lots of files in addition to your sockets? Python open() will bail out at 32768 ope

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
> Besides callLater, a higher level construct to do this is > twisted.internet.task.cooperate(). > https://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-cooperative- > multitasking-with-twisted-getting-things-done-concurrently-11-3352182 is a > presentation dreid did, which should really be tur

Re: [Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Itamar Turner-Trauring
On Mon, 2011-11-14 at 04:07 -0800, Tobias Oberstein wrote: > I am testing a WebSocket based pubsub system .. I have 2 questions .. any > hints welcome! > > Environment: > + FreeBSD 8.2 i386 > + Python 2.7.2 32-bit > + Twisted Trunk > + new kqueue() reactor > > The FreeBSD is tuned for ma

[Twisted-Python] data dispatch on massive connection counts

2011-11-14 Thread Tobias Oberstein
I am testing a WebSocket based pubsub system .. I have 2 questions .. any hints welcome! Environment: + FreeBSD 8.2 i386 + Python 2.7.2 32-bit + Twisted Trunk + new kqueue() reactor The FreeBSD is tuned for massive connection numbers. I can connect 50k WS clients. Fine. 1) Massive dis

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Phil Mayers
On 14/11/11 11:55, Tim Allen wrote: > TL;DR: Github users have expectations of how to interact with projects. > Twisted can make it easy for Github users to contribute by meeting those > expectations, switching from SVN to Git, and changing the UQDS to match. > Keeping things the way they are and

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Tim Allen
On Mon, Nov 14, 2011 at 12:22:59AM -0500, Glyph wrote: > Tim: you're right that there isn't much interest among the current > Twisted core development team. But the whole point of having a Git > mirror (and a presence on Github, which I think is the next step) is > to attract new developers to thi

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Tim Allen
On Sun, Nov 13, 2011 at 09:20:46PM -0500, Glyph Lefkowitz wrote: > On Nov 13, 2011, at 1:13 AM, Tim Allen wrote: > > Should I update the "GitMirror" page to suggest people run "git svn > > clone -rHEAD" instead of using the no-longer-updated official mirror? > > Perhaps the page should also be rena

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread gelin yan
placing in Github is almost a must for attracting developers... At least, it is easy to check the codes... On Mon, Nov 14, 2011 at 4:43 PM, Tobias Oberstein < tobias.oberst...@tavendo.de> wrote: > > But the whole point of having a Git mirror (and a presence on Github, > which I think is the next

Re: [Twisted-Python] Twisted Project Jobs Volunteer

2011-11-14 Thread Tobias Oberstein
> But the whole point of having a Git mirror (and a presence on Github, which I > think is the next step) is to attract new developers to this community. An official presence on Github would be definitely desirable I think, and even more than having a Git mirror technically. Not only attracting