Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[Tom Davis]

On Tue, Mar 22, 2011 at 8:17 PM, Jessica McKellar <
jessica.mckel...@gmail.com> wrote:

> Intrepid Twisted developers and users: from Cambridge, Massachusetts I
> am pleased to announce the arrival of Twisted 11.0.0 pre-release 1.
>
> Tarballs for the pre-release are available at:
>  http://twistedmatrix.com/~jesstess/11.0.0pre1/
>
> Highlights include:
>
>  * a new templating system in Twisted Web, "twisted.web.template",
> derived from Divmod Nevow.
>
>  * improved behavior of subprocess spawning on FreeBSD.
>
>  * an API for constructing endpoints from descriptive strings.
>
>  * twisted.plugin no longer emits a confusing traceback when it can't
> write a cache file.
>
> For more information, see the NEWS file.
>
> Download the tarballs and test away!
>

Awesome! Thanks for doing the release, Jessica! :)


>
> Thanks,
> Jessica
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Deferred documentation.

[Glyph Lefkowitz]

On Mar 21, 2011, at 9:30 PM, Jasper St. Pierre wrote:

> On IRC, exarkun, glyph, spiv and idnar encouraged me to do a bit of
> work on the Defer documentation.

Yay!  This documentation could definitely use some work.

> I kept get confused between the things like returning a Deferred from a 
> callback and chainDeferred, which I found out wasn't that useful:

Yeah, chainDeferred is not a great method.  Now that Deferreds are 
non-recursive, I think it's purely worse than inserting an additional Deferred 
as a result from a callback.

> My eventual goal is to reduce the number of documentation about defer
> down to a near-impossible two documents. I'm hoping to merge some of
> the good stuff of the other thousands of documents.

That would be absolutely great.

> Thoughts so far?

While I applaud your intent, these drafts look quite rough.  The random 
interjections and asides in 
 seem distracting and 
confusing to me.  Trying to put myself in the mind of a newcomer, I find myself 
asking many questions which are irrelevant to what I'm trying to learn:

What's "async"?  Why is it hard?  (The original document mentions asynchronous 
stuff, but in the context of full english sentences.)
Where are we going shopping?  What does shopping have to do with this?
What's gevent?  Does this have something to do with Twisted?
What's node.js?  This looks like Javascript, what does it have to do with 
Twisted, which I thought was in Python?

(Now I've gotten distracted and I'm reading about gevent and node.js rather 
than making my Twisted application work and completing the Deferred tutorial.  
Epic fail.  But, if I were to continue...)

What's "this pattern"?  Functions?  Don't lots of programs use functions?
How do they use it?  Why is it relevant?
Why is Twisted's right hand blue?  (Forget about being a beginner: I honestly 
don't even get this reference.  Googling seems to suggest it has something to 
do with symptoms of heart disease and doesn't seem funny or relevant at all.)
Why is the first explanation of what a Deferred is referred to as "technical 
mumbo-jumbo"? Is this really complicated?  If I am not super good at 
programming already, should I not be reading this?
What's an "operation"?  Does that mean 'function' or 'method' or some other 
special thing?  It says "most operations in Twisted return a Deferred"; but 
I've called lots of functions in Twisted which returned other objects before 
reading this tutorial, or returned None.  Were those actually Deferreds?
Why do I "not know where this Deferred has been"?  Do Deferreds get dirty or 
broken somehow when I add multiple callbacks?  Should I avoid that?

The Python examples in the current Deferred Reference are mostly runnable.  The 
ones that aren't, should be.  The documentation should stress that you can run 
these examples simply, and encourage the reader to download and experiment with 
them, and modify them to see what happens when they do things in a different 
order.

Instead, the "fixup" changes the first example to rely on a fake library, which 
will raise exceptions if I try to run it, but doesn't actually explain that 
'magiclib' isn't real.  This isn't a huge problem in and of itself (it is 
trying to demonstrate the "wrong" way to do things, after all) but it sets up 
the expectation that the rest of the examples are fake, too, and I shouldn't 
bother to run them.

I think the original document has plenty of issues, but these changes look like 
they've been written for people who already mostly understand Deferreds, but 
are having trouble catching some of the nuances, and need humor to diffuse 
their frustration and more examples to illustrate different usage patterns, 
rather than a fundamentally clearer or better explanation than what was offered 
before.  That makes sense, since based on what you've said on #twisted, that's 
basically the position you find yourself in :).  This document is supposed to 
be a tutorial though, explaining how to use Deferreds to users who really have 
no idea what they are (despite its unfortunate name, "Deferred Reference" - 
that should probably be changed).

One thing I think is very good about this attempted rework, though, is the 
explanation of the motivation for having Deferreds at all, before explaining 
how they work.  In the current documentation, it's very unclear why we have 
such an object in the first place, or what the alternatives to it are.  
However, the example presented makes it seem as though you really don't need 
Deferreds, because the only problem with the single-callback approach is 
handling errors.  Another major motivation is the ability to return a Deferred 
through a system with several layers, changing the return value at each layer 
by post-processing it a bit.  (One possible example: a REST API that wants to 
deal with objects, and goes via a translation of [bytes from HTTP]->[JSON 
dicts/lists from parsing those by

Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[Glyph Lefkowitz]

On Mar 22, 2011, at 9:40 PM, Tim Allen wrote:

> On Tue, Mar 22, 2011 at 08:17:25PM -0400, Jessica McKellar wrote:
>>  * a new templating system in Twisted Web, "twisted.web.template",
>> derived from Divmod Nevow.
> 
> Oh boy! Are there pre-release API or other docs for this feature, or
> will I have to download the tarball to read the code and/or wait for
> 11.0.0 to be released to find out about it?

Here's a link which will expire shortly:

.

Please enjoy it while it lasts.

-glyph


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[Tim Allen]

On Tue, Mar 22, 2011 at 08:17:25PM -0400, Jessica McKellar wrote:
>  * a new templating system in Twisted Web, "twisted.web.template",
> derived from Divmod Nevow.

Oh boy! Are there pre-release API or other docs for this feature, or
will I have to download the tarball to read the code and/or wait for
11.0.0 to be released to find out about it?

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[exarkun]

On 12:17 am, jessica.mckel...@gmail.com wrote:

Intrepid Twisted developers and users: from Cambridge, Massachusetts I
am pleased to announce the arrival of Twisted 11.0.0 pre-release 1.

Tarballs for the pre-release are available at:
�http://twistedmatrix.com/~jesstess/11.0.0pre1/

Highlights include:

�* a new templating system in Twisted Web, "twisted.web.template",
derived from Divmod Nevow.

�* improved behavior of subprocess spawning on FreeBSD.

�* an API for constructing endpoints from descriptive strings.

�* twisted.plugin no longer emits a confusing traceback when it can't
write a cache file.

For more information, see the NEWS file.

Download the tarballs and test away!


Hooray!  I just upgraded my mail server to this prerelease and sent this 
email using it. :)


Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[Glyph Lefkowitz]

On Mar 22, 2011, at 8:53 PM, Glyph Lefkowitz wrote:

>>  * an API for constructing endpoints from descriptive strings.
> 
> Actually this feature was in 10.2; I think you meant that the "twistd mail" 
> plugin now uses this functionality.

(I should note that I actually reviewed a draft of this before it went out, but 
my eyes just slid over the mistake.  So the error is more mine than Jessica's!)

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[Glyph Lefkowitz]

On Mar 22, 2011, at 8:17 PM, Jessica McKellar wrote:

> Intrepid Twisted developers and users: from Cambridge, Massachusetts I
> am pleased to announce the arrival of Twisted 11.0.0 pre-release 1.

Hooray!

Thanks for volunteering to do the release, Jessica.  I was glad when Jonathan 
offered to do it again earlier in the month, but I'm even gladder to have new 
people get familiar with the release infrastructure.

> Tarballs for the pre-release are available at:
>  http://twistedmatrix.com/~jesstess/11.0.0pre1/
> 
> Highlights include:
> 
>  * a new templating system in Twisted Web, "twisted.web.template",
> derived from Divmod Nevow.
> 
>  * improved behavior of subprocess spawning on FreeBSD.
> 
>  * an API for constructing endpoints from descriptive strings.

Actually this feature was in 10.2; I think you meant that the "twistd mail" 
plugin now uses this functionality.

>  * twisted.plugin no longer emits a confusing traceback when it can't
> write a cache file.
> 
> For more information, see the NEWS file.
> 
> Download the tarballs and test away!


It Works For Me™ :)

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Announcing Twisted 11.0.0pre1!

[Laurens Van Houtven]

Free Belgian beer for all! (and Corenwyn for Allen)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Announcing Twisted 11.0.0pre1!

[Jessica McKellar]

Intrepid Twisted developers and users: from Cambridge, Massachusetts I
am pleased to announce the arrival of Twisted 11.0.0 pre-release 1.

Tarballs for the pre-release are available at:
 http://twistedmatrix.com/~jesstess/11.0.0pre1/

Highlights include:

 * a new templating system in Twisted Web, "twisted.web.template",
derived from Divmod Nevow.

 * improved behavior of subprocess spawning on FreeBSD.

 * an API for constructing endpoints from descriptive strings.

 * twisted.plugin no longer emits a confusing traceback when it can't
write a cache file.

For more information, see the NEWS file.

Download the tarballs and test away!

Thanks,
Jessica

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] AMP Client disconnecting from server on high memory usage

[Dan-Cristian Bogos]

Hey Guys,

Some issue I've met recently when experimenting with AMP.
Have created an AMP client connecting to a server and stay there
connected (using tap services).
The issues I am getting is that as soon as I load some more serious
data into twisted process (about 700MB on a virtual machine with 3GB
of ram available) the AMP Client does not longer stay connected to the
server. The data is loaded at startup using another service (loading
data by simulating it - adding random strings to some tuple).  I am
using twisted 10.2 out of pypi packages.
I should mention that with small amount of data in memory the
connection stays stable just fine. In logs there is nothing except
seeing the client connecting and disconnecting in the same second.

Any of you know any memory configuration somewhere in the code?

Thanks in advance for any kind of tip!

DanB

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Deferred documentation.

[Itamar Turner-Trauring]

On Mon, 2011-03-21 at 21:30 -0400, Jasper St. Pierre wrote:

> The former is a tutorial that I tried to make informal as possible,
> and the latter is a fixup and rewrite of defer.xhtml stuff to make it
> a bit less, uh, dense.

I like the fixup, it's a good start, especially explaining the
motivation. We can probably drop the "generating deferreds" and
"deferreds in depth" howtos too by merging in any minor details they
mention that this doesn't. Some parts are still cumbersome, e.g.
"However, a Deferred is not a token that allows you to get a specific
result back. You can only get the result by adding callback. This is
because you can add..."



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Deferred documentation.

[Jasper St. Pierre]

(I screwed up my mailman preferences accidentally, so this is
formatted a bit haphazard from copy/paste)

> Thank you for tackling the impossible.

Thank you for the encouragement. I'm curious if you read both
documents, and if you had a preference toward one or the other. I
hopefully only want to have one "definitive deferred brief" in the
end, so a consensus on which road I should take would be would be
helpful:

* -Rewrite has a personal style: first-person perspective, relaxed
language, to-the-point-but-silly examples.

* -Fixup is a sort of gentler working of the current deferred doc that
tries to be a bit more gentle about what the user knows: I don't shove
the user full of an example involving some intricate intertwining of
Deferred.callback and reactor.callLater to introduce them to
callbacks, I instead opt to showcase the built-in HTTP client.

> I remember when reading the old docs the first time that one thing
> bothered me most and this was 'if I get a deferred back, then I'm kind
> of responsible for that beast' so what do I do with it besides adding
> call/errbacks'? I had a hard time and quite a few hours with the
> debugger to figure out the life cycle of a deferred. I also read
> gendefer.html over and over again while trying to figure out the
> relation of a deferred to the reactor, because somehow it must be that
> reactor.run() loop which in the end triggers either call- or errbacks.
>
> This might be typical questions from someone who grew up without garbage
> collection. My problem is, that for coding in a confident way I have to
> have a grasp of the innards of the system.
>
> Summed up
> - How does the life cycle of a deferred look like?
> - Who is responsible for a deferred?
> - If deferreds are related to the reactor, then how are they related?

An alternate goal of my efforts is to have a "rule list". Important
things about the guarantees Deferreds should be documented:

*If* you add more than one callback, *then* the return value of the
last callback will be used as the result for the next callback.
*If* you return a Deferred from a callback, *then* the callback chain
waits until the second Deferred is complete.

> Reading your two docs answers those questions I had back then,
> definitely an improvement.
>
> Minor glitches:
>
> link to node.js is >> http://nodejs.org/

I'm going to have to find new libraries anyway: gevent takes an
eventum-style approach that uses monkey-patches on urllib. I've seen
the callback approach before in a lot of other, different JavaScript
libs, hell, even the DOM-based addEventListener uses this a lot. I
just need to find a Python-based strawman to knock down.

> Thanks again, Werner

Thank you so much for the encouragement.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Deferred documentation.

[Werner Thie]

On 3/22/11 2:30 AM, Jasper St. Pierre wrote:
> On IRC, exarkun, glyph, spiv and idnar encouraged me to do a bit of
> work on the Defer documentation. I kept get confused between the
> things like returning a Deferred from a callback and chainDeferred,
> which I found out wasn't that useful:
>
>   I suppose it's useful for "forking" a deferred
>   you have an "existing" deferred
>   and you want a "new" deferred that starts with the value from
> the end of the existing deferred's callback chain
>   but you don't want the rest of the existing deferred's
> callback chain to be affected by the new deferred's callback chain
>   that's still not really a use case though, just a description
> of the situation that must arise in the use case
>   I guess I should search for actual uses of chainDeferred
>
> Well, I was getting frustrated myself, and I suggested to fix the docs
> once and for all. People pointed me to three documents, all of which
> felt way too dense for me:
>
> http://twistedmatrix.com/documents/current/core/howto/defer.html
> http://twistedmatrix.com/documents/current/core/howto/gendefer.html
> http://twistedmatrix.com/documents/current/core/howto/deferredindepth.html
>
> Jessica also pointed me to http://twistedmatrix.com/trac/ticket/3943,
> which admittedly I haven't read yet.
>
> So, I started writing. I opened up a session of emacs, and produced two 
> things:
>
> http://magcius.mecheye.net/twisted/DeferHowTo-Rewrite.html
> http://magcius.mecheye.net/twisted/DeferHowTo-Fixup.html
>
> Sources:
>
> http://magcius.mecheye.net/twisted/DeferHowTo-Rewrite.rst
> http://magcius.mecheye.net/twisted/DeferHowTo-Fixup.lore
>
> The former is a tutorial that I tried to make informal as possible,
> and the latter is a fixup and rewrite of defer.xhtml stuff to make it
> a bit less, uh, dense.
>
> My eventual goal is to reduce the number of documentation about defer
> down to a near-impossible two documents. I'm hoping to merge some of
> the good stuff of the other thousands of documents.
>
> Thoughts so far?
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Thank you for tackling the impossible.

I remember when reading the old docs the first time that one thing 
bothered me most and this was 'if I get a deferred back, then I'm kind 
of responsible for that beast' so what do I do with it besides adding 
call/errbacks'? I had a hard time and quite a few hours with the 
debugger to figure out the life cycle of a deferred. I also read 
gendefer.html over and over again while trying to figure out the 
relation of a deferred to the reactor, because somehow it must be that 
reactor.run() loop which in the end triggers either call- or errbacks.

This might be typical questions from someone who grew up without garbage 
collection. My problem is, that for coding in a confident way I have to 
have a grasp of the innards of the system.

Summed up
- How does the life cycle of a deferred look like?
- Who is responsible for a deferred?
- If deferreds are related to the reactor, then how are they related?

Reading your two docs answers those questions I had back then, 
definitely an improvement.

Minor glitches:

link to node.js is >> http://nodejs.org/

Thanks again, Werner


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python