Re: [Twisted-Python] txtorcon 0.18.0

2017-01-11 Thread meejah 
Glyph Lefkowitz  writes:

> Trying to figure out if this is a profound dissertation on the nature
> of OPSEC or just an email client crashing...

"yes".

Well, actually ... I use emacs (Gnus) for email and have *nearly entirely*
switched over to tmux for all my terminal multiplexing needs (it's
great!). Except there's that one machine that still uses "screen"
instead, and I "somehow" got confused about prefix-keys and chorded out
the "send the email" Morse-code instead of the "insert a file" one...

-- 
meejah

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] treq POST abborting with: err: ('Could not adapt', '{"....", "..."} )

2017-01-11 Thread Tom Most 
Sure, I'd be happy to help (but won't get to it until the weekend).
Username is twm.

Thanks,
Tom

On 01/11/2017 04:44 PM, Glyph Lefkowitz wrote:
>
>> On Jan 10, 2017, at 6:43 PM, Tom Most > > wrote:
>>
>> On 01/08/2017 06:49 PM, Glyph Lefkowitz wrote:
>>>
 On Jan 6, 2017, at 11:03 AM, steven meiers 
 wrote:

 hi,


 sending a simple post request with a body (cookies) results in a error
 here.

 im using python 3.x with the latest twisted and treq installed via pip
 in a virtualenv.
>>>
>>> Something that we discussed on IRC but did not come up in the
>>> mailing list thread may be the one issue that *should* be fixed in treq:
>>>
>>> Because of this issue, this example
>>> - http://treq.readthedocs.io/en/latest/#post - which the OP was
>>> following, doesn't work on Python 3.
>>>
>>> We should fix the docs so that it does.
>>
>> I submitted a PR to fix this a while ago:
>> https://github.com/twisted/treq/pull/152
>>
>> Also, the docs on RTD are still out of date. Who owns them? I'd be
>> happy to help.
>
> Thanks, hopefully the next time my periodic review comes around I'll
> catch that one.
>
> Looks like I own the docs.  Builds are apparently failing - should I
> add you to the project?
>
> -glyph
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Testing Strategies (was Re: Streaming Requests)

2017-01-11 Thread Glyph Lefkowitz 

> On Jan 9, 2017, at 4:13 AM, Jean-Paul Calderone  
> wrote:
> 
> On Mon, Jan 9, 2017 at 4:52 AM, Glyph Lefkowitz  > wrote:
> On Jan 8, 2017, at 4:34 PM, Jean-Paul Calderone  > wrote:
>> 
>> Here's one example I know of off the top of my head, 
>> >  
>> >.
>>   This one isn't a from-scratch re-implementation of the implicit Request 
>> interface but a Request subclass, instead.  However, it still interacts with 
>> a lot of important pieces of Request which aren't part of IRequest.
> 
> Mark already addressed how he won't be breaking this use-case (which is 
> hugely important, and core to the whole idea of a compatibility policy, so 
> that is as it should be).
> 
> However, this kind of test-mocking is, I think, ultimately done at the wrong 
> layer.  It's trying to override some very vaguely-specified internals in the 
> middle of an implementation.
> 
> 
> Absolutely.

Yay! Glad to have some consensus on this.

> Really, twisted.web should provide its own testing tools, of course.  But if 
> you're going to implement something that does this sort of overriding, I 
> think the idiom to follow would be treq.testing: 
> https://github.com/twisted/treq/blob/fcf5deb976c955ca6ef6484f414d25839932940e/src/treq/testing.py
>  
> ,
>  rather than any of the various implementations of DummyRequest (including 
> more than a few I'm sure I've written).
> 
> Though, note, the link I gave above was support code for something very 
> similar to this treq code: 
> https://github.com/ScatterHQ/flocker/blob/master/flocker/restapi/testtools.py#L460
>  
> 
> 
> To clarify your point a bit (I think):
> 
> MemoryAgent (from Flocker) provides a testing IAgent by implementing an 
> IRequest that does in-memory resource traversal to dispatch requests and 
> generate responses.
> RequestTraversalAgent (from treq) provides a testing IAgent by implementing 
> (using) iosim to do in-memory protocol/transport interacts to drive an 
> in-memory HTTP conversation that runs all of the regular Twisted Web HTTP 
> processing machinery.
> RequestTraversalAgent's approach is better because the protocol/transport 
> interface is better defined.  Because it's better defined, 
> RequestTraversalAgent doesn't have to touch pieces that we might want to 
> consider implementation details (whether they're _-prefixed or not).  It also 
> invokes a larger portion of the real implementation code making it more 
> likely to be a realistic simulation of real-world use of the code.

100% agree.

> Having spelled this out, what occurs to me now is that RequestTraversalAgent 
> is really just a step or so up the ladder and there's further to go.

I generally agree with this next bit, but I have my own spin.  So, one point at 
a time:

> For example, RequestTraversalAgent only needs to exist at all because `iosim` 
> has a distinct interface.  This distinct interface means you need a 
> RequestTraversalAgent-like thing for each reactor-using thing for which you 
> want to provide a test double.

I think that there are pieces of RequestTraversalAgent for which this is true; 
the parts where it has to instantiate its own FakeTransport objects and 
IPv4Addresses and whatever.  This could be streamlined a lot more.

But at the higher level, each layer will want its own public API entrypoint, to 
facilitate the orchestration and setup of domain-specific client/server objects 
and reduce boilerplate for testing at a given level.

> If this adaption behavior were bundled up differently then I think we'd get a 
> lot more in-memory test doubles for free (or closer to it - we'd be another 
> rung up the ladder, at least).

This I definitely agree with.  Each abstraction-level-specific fake should be a 
paper-thin layer over the one below, and each abstraction layer should cover a 
fair amount of ground for itself.  For example, you can test Klein apps 
trivially with treq.testing, without adding very much extra logic at all.

> That seems like it would be a big win given how few of these testing helpers 
> Twisted has historically managed to provide (suggesting it's just too hard to 
> do so given the current tools).

Here I think there are a few mitigating factors:

I don't actually think that it's that hard using current tools.  It's pretty 
easy, in fact, to build up from the bottom and layer subsequent things on top 
of another.
The way I see the problem that we're facing is that we realized we should be 
doing this many years after doing the actual implementation,

Re: [Twisted-Python] txtorcon 0.18.0

2017-01-11 Thread Glyph Lefkowitz 

> On Jan 11, 2017, at 6:58 PM, meejah  wrote:
> 
> 
> 
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Trying to figure out if this is a profound dissertation on the nature of OPSEC 
or just an email client crashing...

-glyph
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] txtorcon 0.18.0

2017-01-11 Thread meejah 


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

txtorcon 0.18.0 is released, improving error-reporting when you have
SAFECOOKIE or COOKIE authentication turned on but can't read the file.

   * https://github.com/meejah/txtorcon/issues/200

You can download the release from PyPI or GitHub (or of
course "pip install txtorcon"):

   https://pypi.python.org/pypi/txtorcon/0.18.0
   https://github.com/meejah/txtorcon/releases/tag/v0.18.0

Releases are also available from the hidden service:

   http://timaq4ygg2iegci7.onion/txtorcon-0.18.0.tar.gz
   http://timaq4ygg2iegci7.onion/txtorcon-0.18.0.tar.gz.asc

You can verify the sha256sum of both by running the following 4 lines
in a shell wherever you have the files downloaded:

cat

[Twisted-Python] txtorcon 0.18.0

2017-01-11 Thread meejah 



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] understanding twisted, better error handling

2017-01-11 Thread Glyph Lefkowitz 

> On Jan 10, 2017, at 3:19 PM, steven meiers  wrote:
> 
> Am Montag, den 09.01.2017, 23:12 -0800 schrieb Glyph Lefkowitz:
>> 
>> 
>>> On Jan 9, 2017, at 5:20 PM, steven meiers 
>>> wrote:
>>> 
>>> the error does come up when you give agent.request a "GET" instead
>>> of a
>>> b"GET".
>> 
>> That's expected; you do have to pass the method as `bytes`, as
>> documented here: https://twistedmatrix.com/documents/16.6.0/api/twist
>> ed.web.iweb.IAgent.html#request
>> 
>> 
> True, but since python 2.x is still shipped with debian testing for
> example there will be some people running into this issue.
> Just to make twisted more approachable this could he handled.

I think that Tristan's suggestion is the best way to address this.

-glyph


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] treq POST abborting with: err: ('Could not adapt', '{"....", "..."} )

2017-01-11 Thread Glyph Lefkowitz 

> On Jan 10, 2017, at 6:43 PM, Tom Most  wrote:
> 
> On 01/08/2017 06:49 PM, Glyph Lefkowitz wrote:
>> 
>>> On Jan 6, 2017, at 11:03 AM, steven meiers < 
>>> commercial...@yahoo.de 
>>> > wrote:
>>> 
>>> hi,
>>> 
>>> 
>>> sending a simple post request with a body (cookies) results in a error
>>> here.
>>> 
>>> im using python 3.x with the latest twisted and treq installed via pip
>>> in a virtualenv.
>> 
>> Something that we discussed on IRC but did not come up in the mailing list 
>> thread may be the one issue that *should* be fixed in treq:
>> 
>> Because of this issue, this example -  
>> http://treq.readthedocs.io/en/latest/#post
>>   - which the OP was following, 
>> doesn't work on Python 3.
>> 
>> We should fix the docs so that it does.
> 
> I submitted a PR to fix this a while ago: 
> https://github.com/twisted/treq/pull/152 
> 
> 
> Also, the docs on RTD are still out of date. Who owns them? I'd be happy to 
> help.

Thanks, hopefully the next time my periodic review comes around I'll catch that 
one.

Looks like I own the docs.  Builds are apparently failing - should I add you to 
the project?

-glyph

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] understanding twisted, better error handling

2017-01-11 Thread steven meiers 
Am Montag, den 09.01.2017, 23:12 -0800 schrieb Glyph Lefkowitz:
> 
> 
> > On Jan 9, 2017, at 5:20 PM, steven meiers 
> > wrote:
> > 
> > the error does come up when you give agent.request a "GET" instead
> > of a
> > b"GET".
> 
> That's expected; you do have to pass the method as `bytes`, as
> documented here: https://twistedmatrix.com/documents/16.6.0/api/twist
> ed.web.iweb.IAgent.html#request
> 
> 
True, but since python 2.x is still shipped with debian testing for
example there will be some people running into this issue.
Just to make twisted more approachable this could he handled.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python