Re: [Twisted-Python] Twisted Release Planning

[Tom Most]



Automating all this is a great project! Please feel free to ping me for reviews.

---Tom

On Thu, Jul 8, 2021, at 10:16 AM, Adi Roiban wrote:
> 
> 
> On Tue, 6 Jul 2021 at 14:21, Adi Roiban  wrote:
>> 
>> 
>> On Tue, 6 Jul 2021 at 01:55, Glyph  wrote:
>>> 
>>> On July 5, 2021 at 3:55:38 PM, Adi Roiban (a...@roiban.ro) wrote:
>>> 
 Hi,
 
 I don't have much time or much "need" but if needed I can help with a new 
 Twisted release.
>>> Hooray! I’m hype for 21.7!
>>> 
>> I have created the release ticket and will create the release branch and RC 
>> soon.
> 
> I am planning to create the release branch soon.
> 
> I plan to do the release based on the updated documentation from 
> https://github.com/twisted/twisted/pull/1616
> 
> That is, create the release branch, push the packages to PyPI and only after 
> these steps as for a review for the PR
> and send the release candidate announcement.
> 
> Let me know if the release pull request should have an approved review before 
> pushing the release candidate files to PyPi.
> 
> -- 
> Adi Roiban
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] IRC still alive?

[Tom Most]

Hi Adi,

IRC has moved to Libera.Chat  due to the Freenode 
implosion . The channel names are the same 
--- #twisted, #twisted-dev, #twisted.web.

---Tom

On Mon, Jul 5, 2021, at 3:45 PM, Adi Roiban wrote:
> Hi,
> 
> Only now I had time to check the Twisted IRC channel and it looks like the 
> channels are no longer registered ... and the IRC bot is not recording the 
> logs.
> 
> Should we continue to use IRC or move to gitter or something else that has 
> history and offline messages by default?
> 
> Cheers
> --
> Adi Roiban
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Strange recursion error with twisted.web

[Tom Most]

On Tue, Jun 15, 2021, at 2:46 AM, Peter Westlake wrote:
> On Tue, 15 Jun 2021, at 01:58, Wim Lewis wrote:
> > On Mon, Jun 14, 2021 at 05:25:42PM +0100, Peter Westlake wrote:
> > > Do we have any idea when the fix for "RecursionError with synchronous 
> > > deferreds (#1549)" will appear in a release?
> > 
> > It's been merged to trunk, so it'll be in the next release... whenever 
> > that is. Twisted tends to release about once or twice a year but I don't 
> > think there's a schedule.
> > 
> > Are you able to test against a recent development version? It's rare for 
> > trunk to have regressions, so it should be pretty safe to run against a 
> > recent version if it fixes a crash you're having.
> 
> I can certainly try. Are development versions available for 
> installation, or do I need to build from source?

We don't publish snapshots, but you can install directly from GitHub, like:

pip install https://github.com/twisted/twisted/archive/trunk.zip

It's probably wise to specify a commit rather than reference trunk. For 
example, to install the latest commit as of now:

pip install 
https://github.com/twisted/twisted/archive/4edc214ce9ca61950dd613b4d2dbb6fa8df28efd.zip

Pip can also install from a Git URL, but using the archive URL is faster 
because you avoid doing a full clone.

---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 21.2.0 Release Announcement

[Tom Most]

Hurrah! Thanks to everyone who contributed!

On Sun, Feb 28, 2021, at 2:27 AM, Craig Rodrigues wrote:
> On behalf of Twisted Matrix Laboratories, I am honored to announce the
> release of Twisted 21.2.0!
> 
> There are two major announcements for this release:
> 
> - *Python 2.7 support has been dropped*.  Twisted 21.2.0 supports Python 
> 3.5.3 and higher only
> - *This will be the last Twisted release to support Python 3.5*.
> 
> Twisted 21.2.0 brings the following:
> - twist dns --pyzone now works on Python 3
> - twisted.web.twcgi now works on Python 3
> - all tests pass on all platforms on Python 3.8, including asyncio tests on 
> Windows
> - tests pass on Python 3.9 (thanks to Michał Górny)
> - type hints, as specified in PEP 484, are now used in functions throughout 
> the codebase
> - mypy (static type checker) is now run over the codebase during CI to help 
> detect problems, and help improve the use of types
> - code in is now formatted with Black (thanks to Tom Most)
> - many CI and automation improvements (thanks to Adi Roiban and Thomas 
> Grainger)
> - many more fixes (225 tickets closed!)
> 
> You can install by running this command:
> 
>   python -m pip install Twisted==21.2.0
> 
> The full tarball is available at:
> https://pypi.org/project/Twisted/21.2.0/#files
> The full NEWS file with all changes is at:
> https://github.com/twisted/twisted/blob/twisted-21.2.0/NEWS.rst
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Logging and SpawnProcess

[Tom Most]

I'm not entirely following the question (is the log output intermixed with 
other data on stdout?) but I think that you might find the implementation of 
twisted.internet.endpoints.ProcessEndpoint a useful example. It relays stderr 
to the logging stream: 
https://github.com/twisted/twisted/blob/c8064075a207af1fc9ce19d8f885a986bc5ab00c/src/twisted/internet/endpoints.py#L336-L479

---Tom

On Wed, Dec 30, 2020, at 11:38 AM, Robert DiFalco wrote:
> Can anyone point me to some sample code for handling logging with 
> SpawnProcess? Right now I'm using an EchoProtocol that takes stdout and 
> resends it to the log. But it's kludgy because some sub-processes use 
> stdin/stderr for marshaling data. What I'd most like to do is separate data 
> transfer from log output in a clean way. 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Current twisted dns client just doesn't work

[Tom Most]

Yes, that would be an improvement on the current behavior, though the ideal 
behavior is to query both A and  simultaneously, returning the union of 
their results.

---Tom

On Sat, Dec 5, 2020, at 2:06 PM, spam tam wrote:
> Ok. Instead ANY I can query A and if A doesn't provide IP then query 
> 
> On Sat, Dec 5, 2020 at 5:07 AM Tom Most  wrote:
> >
> > Why do an ANY query at all? As Glyph noted[1] the correct solution is to 
> > query A and  concurrently. If you want a temporary hack wouldn't the 
> > most straightforward thing be to query just A instead of ANY + A?
> >
> > ---Tom
> >
> > [1]: https://twistedmatrix.com/trac/ticket/9691#comment:5
> >
> > On Fri, Dec 4, 2020, at 5:37 PM, spam tam wrote:
> > > Twisted provide ability to run dns client in the same thread as main
> > > twisted application. But it just doesn't work.
> > >
> > > I created pull request which fix the problem:
> > > https://github.com/twisted/twisted/pull/1488
> > >
> > > I would like to know if twisted community knows about it. This client
> > > doesn't work for years. I think that we should fix the problem.
> > >
> > > Twisted dns client should work. What do you think? I think we should
> > > discuss the problem. DNS client protocol provide ANY, A, 
> > > requests. Current dns ANY request is not enough. My code add A dns
> > > request after current ANY request.
> > >
> > > Is this enough fix or anybody think it's not enough or may be we
> > > should find another solution. I think twisted framework should work
> > >
> > > ___
> > > Twisted-Python mailing list
> > > Twisted-Python@twistedmatrix.com
> > > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> > >
> >
> > ___
> > Twisted-Python mailing list
> > Twisted-Python@twistedmatrix.com
> > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Current twisted dns client just doesn't work

[Tom Most]

Why do an ANY query at all? As Glyph noted[1] the correct solution is to query 
A and  concurrently. If you want a temporary hack wouldn't the most 
straightforward thing be to query just A instead of ANY + A?

---Tom

[1]: https://twistedmatrix.com/trac/ticket/9691#comment:5

On Fri, Dec 4, 2020, at 5:37 PM, spam tam wrote:
> Twisted provide ability to run dns client in the same thread as main
> twisted application. But it just doesn't work.
> 
> I created pull request which fix the problem:
> https://github.com/twisted/twisted/pull/1488
> 
> I would like to know if twisted community knows about it. This client
> doesn't work for years. I think that we should fix the problem.
> 
> Twisted dns client should work. What do you think? I think we should
> discuss the problem. DNS client protocol provide ANY, A, 
> requests. Current dns ANY request is not enough. My code add A dns
> request after current ANY request.
> 
> Is this enough fix or anybody think it's not enough or may be we
> should find another solution. I think twisted framework should work
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] towncrier review policies and etiquette (and maybe releases too)

[Tom Most]

I think that it's fine to fall back to the twisted/twisted policies[1] for any 
project in the Twisted GitHub organization. The workflow won't apply directly 
--- no Trac, etc. --- but the core "someone must review all changes" and "test 
all the things" principles are relevant. That's how I treat Treq.

---Tom

[1]: https://twistedmatrix.com/trac/wiki/ContributingToTwistedLabs

On Wed, Oct 21, 2020, at 12:11 PM, Kyle Altendorf wrote:
> Where does towncrier stand on review policies and etiquette?  I 
> generally don't like to just jump into new projects and start reviewing 
> and merging but I don't see other activity on that front nor do I see 
> any guidance on how it should be done.  If I know how to proceed 
> properly then I might dedicate some review time to it.
> 
> Cheers,
> -kyle

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Broken Javascript on https://twistedmatrix.com ?

[Tom Most]

Access-Control-Allow-Origin is a header that would need to be set on 
PayPal/Flattr's side. 

They may have newer versions of these buttons that permit hotlinking, but the 
best solution is probably to host the images locally. That's what's best for 
performance of the page because you avoid extra DNS resolutions and can reuse 
the connection(s) used to load the rest of the page.

---Tom

On Thu, Sep 17, 2020, at 12:47 PM, Craig Rodrigues wrote:
> Doing a force refresh in my browser fixed it.  I can now click on 
> *Publish/Subscribe* and switch the tab.
> 
> I'm still seeing these errors which is causing the Paypal and Flattr gif's 
> not to load:
> 
> Access to image at 'https://www.paypalobjects.com/en_US/i/scr/pixel.gif' from 
> origin 'https://twistedmatrix.com' has been blocked by CORS policy: No 
> 'Access-Control-Allow-Origin' header is present on the requested resource.
> (Access to image at 'https://button.flattr.com/flattr-badge-large.png' from 
> origin 'https://twistedmatrix.com' has been blocked by CORS policy: No 
> 'Access-Control-Allow-Origin' header is present on the requested resource.
> 
> I'm not familiar with CORS, so don't know what the proper fix for this is.
> 
> Thanks for fixing this.
> --
> Craig
> 
> 
> 
> 
> On Thu, Sep 17, 2020 at 11:55 AM Glyph  wrote:
>> This is the JS that I specifically changed, which is no longer being served 
>> by twistedmatrix.com, so just like… reload harder, or something.  Clear your 
>> cookies, zap your caches, restart your browser, repent of your sins, etc.  
>> Maybe repent before clearing the caches so you don't have any cached sins 
>> either.
>> 
>> If you want to work on this yourself, we do have *something* of a local 
>> staging infrastructure set up with... vagrant, I think?, but I haven't tried 
>> it.  If you want to test your changes in developmestuction, check out braid, 
>> edit the HTML and JS in t-web, push to master, and then `fab 
>> config.production t-web.update`.
>> 
>> Good luck,
>> 
>> -g
>> 
>> P.S.: braid still requires python2, so anybody enthusiastic about python3 
>> porting is encouraged to take a look at it :-).
>> 
>>> On Sep 17, 2020, at 8:37 AM, Craig Rodrigues  wrote:
>>> 
>>> Thanks for looking into this.  Now I am seeing this error on the 
>>> https://twistedmatrix.com :
>>> 
>>> jsapi?key=ABQIJE-f7vmwI1-jxkmwRe4lUxTAs18ELyhzmLGaHoc1qJwmpJy3zhR3LMRRdwqg5PSY4tnVO684msUklQ:136
>>>  Uncaught Error: Must call google.charts.load before 
>>> google.charts.setOnLoadCallback
>>> at Object.J.R [as setOnLoadCallback] 
>>> (jsapi?key=ABQIJE-f7vmwI1-jxkmwRe4lUxTAs18ELyhzmLGaHoc1qJwmpJy3zhR3LMRRdwqg5PSY4tnVO684msUklQ:136)
>>> at twisted.js:92
>>> jsapi?key=ABQIJE-f7vmwI1-jxkmwRe4lUxTAs18ELyhzmLGaHoc1qJwmpJy3zhR3LMRRdwqg5PSY4tnVO684msUklQ:141
>>>  Uncaught Error: Module "feeds" is not supported.
>>> at 
>>> jsapi?key=ABQIJE-f7vmwI1-jxkmwRe4lUxTAs18ELyhzmLGaHoc1qJwmpJy3zhR3LMRRdwqg5PSY4tnVO684msUklQ:141
>>> 
>>> 
>>> Where is that coming from?
>>> 
>>> 
>>> --
>>> Craig
>>> 
>>> On Wed, Sep 16, 2020 at 11:28 PM Glyph  wrote:
 We were depending on some Google APIs that were disabled a really long 
 time ago, so I pushed & deployed this commit 
 
  to get the front page back to working.
 
 -g
 
> On Sep 16, 2020, at 2:53 PM, L. Daniel Burr  wrote:
> 
> Hi Craig,
> 
> No error is being emitted to the console when you click on a tab.  There 
> are some cross-origin script failures being logged, and Firefox blocked 
> google urchin, which causes some more error messages.
> 
> As far as I can tell, the JS code that is supposed to hook up the 
> clicking of the tab (an anchor tag with a relative href) is not being 
> executed, either because it is being blocked by the browser, or as a 
> side-effect of other things being blocked.
> 
> Hope this helps,
> 
> Daniel
> --
> L. Daniel Burr
> ldanielb...@me.com
> (312) 656-8387
> 
>> On Sep 16, 2020, at 4:36 PM, Craig Rodrigues  
>> wrote:
>> 
>> If I go to https://twistedmatrix.com and click on
>> one of the tabs in the *Code Examples* section,
>> such as *Publish/Subscribe*, the tab does not switch.
>> 
>> This looks like a Javascript problem.  Does anyone with knowledge
>> of Javascript have an idea what the problem is?
>> 
>> --
>> Craig
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
 
 ___
 

Re: [Twisted-Python] Black enabled in trunk

[Tom Most]

On Tue, Sep 15, 2020, at 7:26 AM, Kyle Altendorf wrote:
> On 2020-09-14 13:43, Glyph wrote:
> 
> >> On Sep 13, 2020, at 11:45 PM, Tom Most  wrote:
> >> To adjust the formatting:
> >> 
> >> tox -e black-reformat
> >> 
> >> The formatting is checked by a new GitHub Actions lint built.
> > 
> > Could we possibly use something like this:
> > 
> > https://github.com/cclauss/autoblack
> > 
> > to just _do_ the formatting rather than "check" if it's correct?  PR 
> > templates are all well and good but the best checklist item is the one 
> > that's already checked off...
> 
> It might be worthwhile but I think there is some downside to CI 
> frequently injecting commits (or amending them).

I'm not opposed to it in principle, but is there a version of this that works? 
That link says "tl;dr: It does not work."

I don't want to develop custom tooling. This project is about moving away from 
that!

> >> Inevitably, changes like this cause conflicts. For small PRs it's 
> >> easiest to merge forward and then run tox -e black-reformat. For 
> >> larger ones it can help to apply formatting before merge. To do this:
> >> 
> >> * Run black on the files your branch changes (be sure to use Black 
> >> 20.8b1, not an older version)
> >> * Commit the result, like `git commit -am "Fade to black"`
> >> * Add that commit to .git-blame-ignore-revs to avoid polluting git 
> >> blame.
> 
> Is .git-blame-ignore-revs going to be the next 'newsfile' what with 
> every PR adding lines to it?  Also, for the record, this doesn't address 
> GitHub blame and does require everyone set the previously mentioned git 
> config since it doesn't go with the repo.  :[  Though that's mostly an 
> annoyance with git.

I don't expect so. There aren't that many PRs large enough to benefit from this 
technique, and large PRs tend to enter the codebase slowly. 

> Oh yeah...  thanks!

You're welcome! 

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Black enabled in trunk

[Tom Most]

Hi all,

Craig and I have been working to land a switch to The *Black* Coding Style 
. It's in! 
The tree has been reformatted 
.
 Some notable changes:

* 3-2-1 blank lines are replaced by PEP 8 style 2-1-1.
* String syntax uses "double quotes".

Run this command to exclude the formatting from git blame 
:

git config blame.ignoreRevsFile .git-blame-ignore-revs

There are two new Tox testenvs. To check style:

tox -e black

To adjust the formatting:

tox -e black-reformat

The formatting is checked by a new GitHub Actions lint built.

There is some follow-up work planned:

* Many PRs have conflicts (more on this below).
* Various wiki pages need updates. I've done a few, but there are probably more.
* The coding standard should be updated 
.
* The PR template needs an update 
.
* A few examples need reformatting  (they weren't 
formatted because they fail to lint).

Inevitably, changes like this cause conflicts. For small PRs it's easiest to 
merge forward and then run tox -e black-reformat. For larger ones it can help 
to apply formatting before merge. To do this:

* Run black on the files your branch changes (be sure to use Black 20.8b1, not 
an older version)
* Commit the result, like `git commit -am "Fade to black"`
* Add that commit to .git-blame-ignore-revs to avoid polluting git blame.
* Merge forward. You can ignore formatting while performing the merge.
* Then run `tox -e black-reformat` to ensure you are in sync. Commit any 
changes it generates.

A big thanks to Hawkowl for kicking this all off and Craig for quick review 
turnaround. May we all be mildly displeased with the formatting, yet never have 
to review it again!

---Tom___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Pycodestyle disabled in trunk, time to enable black?

[Tom Most]

On Mon, Aug 31, 2020, at 10:06 AM, Craig Rodrigues wrote:
> In this PR:
> https://github.com/twisted/twisted/pull/1365
> 
> pycodestyle was disabled in trunk in order to make it easier to merge changes
> and not have pycodestyle error out, due to the code not meeting existing 
> Twisted style guidelines.
> So this means that the old pycodestyle is not enforced in trunk.
> 
> In this PR:
> https://github.com/twisted/twisted/pull/1356
> 
> a `.git-blame-ignore-revs 
> `
>  file was added to the tree.  I guess in future,
> if the entire tree is reformatted, then we can add that git SHA for the PR 
> which reformats
> the tree, so that that revision will be ignored during a `git blame`.
> 
> There is this pending (over 1 year) PR to add black to CI:
> 
> https://github.com/twisted/twisted/pull/1134
> 
> There are some comments in there related to documentation changes.
> Can the non-documentation changes be pulled out of PR 1134 and merged to 
> trunk now?

I created a PR without them here: https://github.com/twisted/twisted/pull/1380

> I guess whenever black is merged to trunk, then a follow-up PR to update 
> `.git-blame-ignore-devs` would be required?

I think merging that PR should involve immediately running tox -e 
black-reformat, isort, committing the result, and adding that commit to 
.git-blame-ignore-revs. Those two commits should be pushed directly to trunk.

They can then proceed to push formatting commits to the open PR branches where 
that is possible. I went through all of these and dealt with the conflicts a 
while back.

I'd like to get this in this weekend.

---Tom___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] DNS custom response (NXDOMAIN)

[Tom Most]

On Wed, Aug 5, 2020, at 3:59 PM, Rob Gormley wrote:
> But I tried with dns.Record_NULL to generate an NXDOMAIN-like response 
> without success. How can I implement this?

You need to fail with AuthoritativeDomainError. The pieces go together 
something like this:

from twisted.names import dns, client, common

class MyResolver(ResolverBase):
def _lookup(self, name, cls, record_type, timeout):
# This says "my resolver can't handle that name, try somewhere else".
# DNSServerFactory will try the caches or recurse to clients.
return defer.fail(dns.DomainError(name))
# This says "that name definitely doesn't exist".
# The client will get NXDOMAIN
return defer.fail(dns.AuthoritativeDomainError(name))


factory = DNSServerFactory(
authorities=[MyResolver()],
caches=[...],
   clients=[client.Resolver()],  # used to recurse
)

---Tom___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] new code: 3.5 or 3.6?

[Tom Most]

On Thu, Jun 25, 2020, at 1:50 AM, Ian Haywood wrote:
> when writing new code for twisted today, can we use 3.6 features or is 
> 3.5 still required? (it's actually a library feature enum.IntFlag I'm 
> curious about, not type annotations)

We'll drop Python 3.5 after the next release, so for now we have to avoid 3.6 
features. This policy may vary for other Twisted projects, though --- Klein has 
already dropped 3.5 support, while Treq still supports 2.7 for now.

---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Running tests with monkeytype

[Tom Most]

On Sat, Jun 20, 2020, at 3:10 PM, Maarten ter Huurne wrote:
> Which also brings up the question: in the long term (*), do we want to 
> have both type annotations and types in the docstring? Or do we only 
> want to document types in docstrings if there is no exact annotations 
> possible in Python's type system?

Let's definitely move to type annotations, since they are much more likely to 
be correct due to validation by MyPy.

To start, we'll need to revise the Twisted coding standard [1] to permit type 
annotations instead of @type lines in the docstring. I think that we should 
suggest, but not require, annotation syntax to start. I filed a ticket [2] for 
this.

---Tom

[1]: 
https://twistedmatrix.com/documents/current/core/development/policy/coding-standard.html#docstrings
[2]: https://twistedmatrix.com/trac/ticket/9859

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Tom Most]

On Fri, May 22, 2020, at 10:54 PM, Glyph wrote:
> So, pidfd's cool, we should totally use it. Also we should use posix_spawn 
> and maybe some other stuff too. But I wonder if there's any heuristic we 
> could use to speed up our current strategy, like ordering the to-reap list by 
> putting things with no open FDs at the front of it? And optimistically 
> assuming that once we've found something to reap, maybe we can stop? And 
> maybe it should run in a cooperator, rather than just blocking the reactor 
> indefinitely?

Those all sound like reasonable optimizations with little downside to me.

> Honestly it had not occurred to me that people were managing 20k+ python 
> interpreters at a time with spawnProcess (although, yikes, amazing, you've 
> gotta talk more about this application, and what kind of hardware you're on!)

Nonono, that was a synthetic stress test. It did spawn Python processes, but 
they immediately called `exec sleep` to conserve memory.

The result of the test was that the service would run out of memory _well_ 
before inefficiencies in Twisted's reaping scheme became a problem.
 
---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Tom Most]

On Tue, May 19, 2020, at 5:41 AM, Jean-Paul Calderone wrote:
> On Tue, May 19, 2020 at 8:33 AM Barry Scott  
> wrote:
>> Why not waitpid for all children (pid=0 or pid=-1) in a single call? 
> 
> Because then you might reap a child process someone launched with a different 
> API - eg the Python subprocess module.

Exactly. I am currently using this as a workaround, but it's not a general 
solution. You need to be careful because subprocess use hides in many places. 
For example, beware uuid.uuid1().

---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Tom Most]

On Sun, May 17, 2020, at 12:22 PM, Glyph wrote:
>> In the one case that may be complicated I think that pinning Twisted will be 
>> acceptable, since the service in question is a process supervisor without 
>> direct external exposure.
> Well in that case I'll just do my best to add lots of goodies (posix_spawn 
> support!) that will make you want to upgrade that to current twisted and 
> python 3.8 :).

Oh it's not for lack of wanting. :-)

The service in question is actually spawning Python processes, so posix_spawn 
support doesn't seem likely to help much: fork() + exec() is a drop in the 
bucket compared to Python imports, particularly in Python 3 (I'm hoping to 
explore oxidized-importer [1] to help with this at some point).

If looking improve process management in Twisted, I think that the 
lowest-hanging fruit is to watch children with pidfd, rather than reaping all 
children on SIGCHLD. This O(n) reaping --- one waitpid() call per running 
process --- starts to consume substantial CPU. IIRC it pegged a core at 20,000 
processes or so.

---Tom

[1]: https://pypi.org/project/oxidized-importer/___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Tom Most]

On Sun, May 17, 2020, at 11:31 AM, Glyph wrote:
> 
>> On May 16, 2020, at 3:12 PM, Tom Most  wrote:
>> I'm (unfortunately) still shipping some Twisted apps on Ubuntu 16.04's 
>> Python 3.5. However, I am fine with dropping Python 3.5 support. I think 
>> it's certainly worth it for variable annotations and contextvars.
> 
> Would your plan to be to just keep twisted pinned until you can EOL ubuntu? I 
> don't want to be putting (non-trivial numbers of) users into the position of 
> having to skip a bunch of updates.
> 
> -glyph

Yes, I'm already working towards EOL of Ubuntu 16.04. I expect to have that 
work completed by the end of the year.

In the one case that may be complicated I think that pinning Twisted will be 
acceptable, since the service in question is a process supervisor without 
direct external exposure.

---Tom___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [RFC] Drop support for Python 3.5 sometime after May 2021?

[Tom Most]

I'm (unfortunately) still shipping some Twisted apps on Ubuntu 16.04's Python 
3.5. However, I am fine with dropping Python 3.5 support. I think it's 
certainly worth it for variable annotations and contextvars.

---Tom

On Fri, May 15, 2020, at 10:56 PM, Glyph wrote:
> 
> 
>> On May 15, 2020, at 8:40 PM, Craig Rodrigues  wrote:
>> 
>> Maybe it would be OK to do one more release of Twisted and announce that as 
>> the last release supporting Python 3.5, before
>> dropping support? 
> Yeah; whenever we drop a Python version we should always support at least one 
> more release, so that people have some notice before they lose access to the 
> next set of security updates.
> 
> Any 3.5 users on this list who would want to postpone it longer than this?
> 
> -glyph
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] IReactorTime.seconds: epoch time or no?

[Tom Most]

I'll offer a dissenting opinion:

I've worked on systems where the reactor is patched to use monotonic time. 
(This is essential on embedded systems that lack a real-time clock.) I'm not 
aware of this causing any issues, and it fixed real problems.

HTTPFactory is quite unusual in assuming that it is epoch time. Most parts of 
Twisted that require wall time call `time.time()` or equivalent directly. IMO 
HTTPFactory is in error here, and this is simply a bug. We should change 
HTTPFactory to use `time.time()` directly.

Since the time base of IReactorTime.seconds() hasn't ever been clearly defined, 
I think that we should really go the other way: use `time.monotonic()` by 
default where possible. (Or, more realistically, make it an option and maybe 
the default.)

---Tom

On Mon, Mar 23, 2020, at 1:24 PM, Amber Brown (hawkowl) wrote:
> 
> 
> On 24/3/20 6:45 am, Richard van der Hoff wrote:
> > HTTPFactory seems to think that `reactor.seconds` is reliably an epoch 
> > time (see 
> > https://github.com/twisted/twisted/blob/trunk/src/twisted/web/http.py#L3110,
> >  
> > etc).
> > 
> > On the other hand. AsyncioSelectorReactor.seconds() returns a monotonic 
> > time:
> > 
> > python3 -c 'from twisted.internet import asyncioreactor; 
> > print(asyncioreactor.AsyncioSelectorReactor().seconds())'
> > 41116.763594412
> > 
> > One of these is wrong... I think it's HTTPFactory making bad 
> > assumptions, but can anyone confirm the intention here?
> 
> 
> It's a bug in asyncioreactor. IReactorTime defines it as "current time 
> in seconds", which _probably_ should be defined nicer 
> (https://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IReactorTime.html#seconds)
>  
> but is generally epoch time.
> 
> asyncioreactor does have a few problems with time:
> 
> https://twistedmatrix.com/trac/ticket/9611
> https://twistedmatrix.com/trac/ticket/9780
> 
> I can't find a bug for this issue, though, although I'm almost sure I've 
> seen it before... so, if you could file one (and maybe a patch to fix 
> it!) then that would be appreciated.
> 
> - Amber
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Inheriting DatagramProtocol, which is an old-style class

[Tom Most]

Old-style classes remain old-style for compatibility reasons. You have some 
options, though:

1. Set the TWISTED_NEWSTYLE environment variable [1] to make all such old-style 
classes new-style.
2. When subclassing it, also subclass object, making the result new-style.
3. Switch to Python 3, where all-classes are new-style.

Option 3 is the best, since we will drop Python 2.7 support soon enough. Option 
1 may be helpful when migrating a codebase.

---Tom

[1]: 
https://github.com/twisted/twisted/blob/c0a51509974e995537212efc5074140388585da6/src/twisted/python/compat.py#L61

On Fri, Feb 7, 2020, at 9:44 PM, Maarten ter Huurne wrote:
> On Saturday, 8 February 2020 01:50:04 CET Go Luhng wrote:
> > I am creating a child class
> > `Child(twisted.internet.protocol.DatagramProtocol)` to implement a
> > custom UDP multicast protocol.
> > 
> > I need to add an `__init__()` to `Child`, but as part of that I need
> > to call `super()` which is impossible because `DatagramProtocol` is
> > old-style.
> > 
> > How should I proceed?
> 
> There is no __init__() in DatagramProtocol or in its superclass 
> AbstractDatagramProtocol, so you can just skip the call.
> 
> > Also, why are there old-style classes in the latest release of
> > Twisted?
> 
> I don't know the reason, but it seems to be deliberate, since 
> AbstractDatagramProtocol is annotated with @_oldStyle in the source.
> 
> Bye,
>   Maarten

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Python 2.7 Plan

[Tom Most]

On Wed, Dec 11, 2019, at 1:06 PM, Wilfredo Sánchez Vega wrote:
>  [snip]
> 
>   Glyph also notes some infrastructure we use that is blocked on code fixes:
> 
>   - Trac: https://trac.edgewall.org/ticket/12130
> Looks like we can leave the Trac instance running in a Twisted 19  WCGI 
> server until they fix that.

Note that multipart requests (relevant for Trac if we want to support ticket 
attachments) are broken on Python 3.7+ due to some questionable changes to the 
cgi module. We may consider this blocking if we aim to transition to 3.7+. 
There is a fix here: https://github.com/twisted/twisted/pull/1207

>   - DNS: https://twistedmatrix.com/trac/ticket/9496
> I'd suggest we do the same as with Trac until this is fixed, but Glyph 
> disagrees.
>
> [snip] I would suggest that our next release is the final release for 2.7.  
> This would require a fix to the DNS issue #9496.

I have a fix for this ready for review here: 
https://github.com/twisted/twisted/pull/1212

---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted tips for designing highly concurrent twisted REST API

[Tom Most]

On Thu, Jul 11, 2019, at 1:46 AM, Scott, Barry wrote:
> On Tuesday, 9 July 2019 22:04:11 BST Tom Most wrote:
> 
> ...snip...
> 
> > The reactor's own thread pool is really for DNS resolution.
> 
> Is that still true in the default case? We are use the twisted code that 
> talks 
> to DNS servers as the threaded resolver adds too much latency.

As far as I know, yes. The higher-level APIs use getaddrinfo() at least.

https://twistedmatrix.com/documents/current/api/twisted.internet._resolver.GAIResolver.html
https://github.com/twisted/twisted/blob/c0776850e756adfcdc179a7fd9e4c8f5cbc4838d/src/twisted/internet/base.py#L1007

TCP6ClientEndpoint also invoke getaddrinfo() directly.

twisted.names is certainly more performance but it's missing some system 
integration features that make it unsuitable as a default:

* No support for the domain or search resolv.conf directives
* No NSS lookups (e.g., systemd integration)

This is all on Linux, YMMV on other platforms.

---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted tips for designing highly concurrent twisted REST API

[Tom Most]

Hi,

There are likely a few things wrong here.

1. You are using requests.get() to make a HTTP request. This is blocking. You 
might consider using Twisted's Agent 
 
API instead (or treq , which puts a 
requests-like API atop Agent).

2. As you add load your long computations will be queued. deferToThread 

 dispatches the long_computation to the reactor's default thread pool 
.
 This poll has a maximum size and will queue work once it has spun up that many 
threads.

Rather than using deferToThread (which we should really deprecate as it doesn't 
accept a reactor parameter...) I'd recommend instantiating your own ThreadPool 

 and using deferToThreadPool 
.
 The reactor's own thread pool is really for DNS resolution. You risk deadlocks 
in a system that ThreadPoolThreadPoolThreadPool

3. The specifics of what long_computation are also important. If it doesn't 
release the GIL you won't get real parallelism (this is a Python thing, not a 
Twisted thing). See this recent thread on the topic 
.

Though the mechanisms differ athis thread on the topicny of the above would 
cause the response time to increase as you add load.

Good luck,
Tom

On Tue, Jun 25, 2019, at 11:51 PM, Waqar Khan wrote:
> 
> Sorry I had a typo in twisted program
> 
> *@defer.inlinecallbacks*
> *def **long_computation*(rec_type, data)*:
** **# some long computation
** *defer.returnValue(recs)**
> @defer.inlinecallbacks
> def fetch_data(user_id):
> r *= yield*json.*loads*(requests.*get*('url/to/fetch/%s'*%**user_id*).text)
>  defer.returnValue(r)
> 
> 
> @defer.inlinecallbacks
> def fetch_recs(user_id):
>  data = yield fetch_data(user_id)
>  recs = {}
>  for stype in similar_types:
> *d = defer.ToThread(long_computation, *(stype, data)) // typo was here*
>  rec = yield d
>  recs[stype] = rec
>  defer.returnValue(recs)
> 
> 
> On Tue, Jun 25, 2019 at 11:48 PM Waqar Khan  wrote:
>> Hello folks,
>>  I recently stumbled upon twisted and was wondering if it could suit my 
>> needs. On one hand, I want to use python but on another hand there are all 
>> these scalability concerns with this language so, I though I would pick the 
>> brains of the community. So.. a flask based app would look something like 
>> this.
>> 
>> similar_types *= *['foo', 'bar', 'baz']
>> 
>> 
>> *def **long_computation*(rec_type)*:
** **# some long computation
** **return *recs
>> 
>> *@app.route*('/fetch_similar_users/'
>> *def **fetch_similar_users*(*user_id*)
>> r *= 
>> *json.*loads*(requests.*get*('url/to/fetch/%s'*%**user_id*).text)
>> recs *= *{}
>> *for *stype *in *similar_types*:
** *recs[stype] *= **long_computation*(rec_type)
>>*return *recs
>> 
>> 
>> Now, I tried to "twistify" but it failed.
>> 
>> *@defer.inlinecallbacks*
>> *def **long_computation*(rec_type)*:
** **# some long computation
** *defer.returnValue(recs)**
>> @defer.inlinecallbacks
>> def fetch_data(user_id):
>> r *= yield*json.*loads*(requests.*get*('url/to/fetch/%s'*%**user_id*).text)
>>  defer.returnValue(r)
>> 
>> 
>> @defer.inlinecallbacks
>> def fetch_recs(user_id):
>>  data = yield fetch_data(user_id)
>>  recs = {}
>>  for stype in similar_types:
>>  d = defer.ToThread(fetch_data, *(stype))
>>  rec = yield d
>>  recs[stype] = rec
>>  defer.returnValue(recs)
>> 
>> 
>> 
>> I wrapped all the above in twisted render_Get method.. but then I did a load 
>> test with locust (https://docs.locust.io/en/latest/what-is-locust.html) 
>> framework.
>> It choked. As the time progressed, the response time increased.
>> I am guessing, things are still blocking. 
>> 
>> Can you please help me look into the right place. Why exactly am I seeing 
>> increase in response time as the time progresses. I am guessing things are 
>> still working in "blocking" fashion but i thought the above should run 
>> things in async.
>> Thanks
>> 
>> 
>> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Agent Python 2 ->3 Porting Pitfalls

[Tom Most]

I filed https://twistedmatrix.com/trac/ticket/9643 and submitted a PR: 
https://github.com/twisted/twisted/pull/1146

---Tom

On Thu, May 23, 2019, at 11:50 PM, Glyph wrote:
> I definitely agree with this!
> 
> -g
> 
> > On May 23, 2019, at 9:08 PM, Tom Most  wrote:
> > 
> > This looks like a straightforward bug to me. I don't see any reason we 
> > can't change the implementation to either raise TypeError immediately or 
> > accept a string, since any code that passes a string doesn't work right 
> > now. Would you file a bug at https://twistedmatrix.com/trac/newticket?
> > 
> > ---Tom
> > 
> > -- 
> >  Tom Most
> >  t...@freecog.net
> > 
> > On Mon, May 20, 2019, at 12:22 AM, Thomas Westfeld wrote:
> >> Dear all,
> >> 
> >> I just migrated some quite basic twisted code from Python 2.7 to Python 
> >> 3. Everything worked quite nicely, the main issue was the correct 
> >> handling of strings vs. bytes.
> >> 
> >> There was a mean error I made when using the Twisted Agent http client: 
> >> I specified the method as a Unicode string, instead of using 
> >> Agent.request(b'GET',b'http://whatever.com'). The mean thing is that 
> >> the code does not produce any error. It connects to the server via TCP 
> >> but then stops.
> >> 
> >> It is given correctly in the examples of the docs and also in the 
> >> source code, but wouldn't it make things easier to make a type check in 
> >> the request method if the method parameter is of type Unicode and raise 
> >> an error?
> >> 
> >> Best regards,
> >> Thomas Westfeld
> >> ___
> >> Twisted-Python mailing list
> >> Twisted-Python@twistedmatrix.com
> >> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> >> 
> > 
> > ___
> > Twisted-Python mailing list
> > Twisted-Python@twistedmatrix.com
> > https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted Agent Python 2 ->3 Porting Pitfalls

[Tom Most]

This looks like a straightforward bug to me. I don't see any reason we can't 
change the implementation to either raise TypeError immediately or accept a 
string, since any code that passes a string doesn't work right now. Would you 
file a bug at https://twistedmatrix.com/trac/newticket?

---Tom

-- 
  Tom Most
  t...@freecog.net

On Mon, May 20, 2019, at 12:22 AM, Thomas Westfeld wrote:
> Dear all,
> 
> I just migrated some quite basic twisted code from Python 2.7 to Python 
> 3. Everything worked quite nicely, the main issue was the correct 
> handling of strings vs. bytes.
> 
> There was a mean error I made when using the Twisted Agent http client: 
> I specified the method as a Unicode string, instead of using 
> Agent.request(b'GET',b'http://whatever.com'). The mean thing is that 
> the code does not produce any error. It connects to the server via TCP 
> but then stops.
> 
> It is given correctly in the examples of the docs and also in the 
> source code, but wouldn't it make things easier to make a type check in 
> the request method if the method parameter is of type Unicode and raise 
> an error?
> 
> Best regards,
> Thomas Westfeld
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Drop support for Python 3.4?

[Tom Most]

I'm happy to drop 3.4 support and prune our build matrix a bit, as well as that 
in downstream projects.

--
 Tom Most
 t...@freecog.net



On Tue, Mar 26, 2019, at 7:59 PM, Craig Rodrigues wrote:
> What do people think of dropping Twisted support for Python 3.4?
> 
> According to https://devguide.python.org/#status-of-python-branches
> 
> Python 3.4 EOL'd on March 19, 2019.
> 
> In the Python 3 world, we have Python 3.4, 3.5, 3.6, 3.7, and at
> the end of this year we will have Python 3.8.
> 
> That's quite a lot of Python versions to support.
> 
> Python 3.5 introduced async/await keywords, which are very relevant to 
> Twisted:
> https://docs.python.org/3.5/whatsnew/3.5.html#whatsnew-pep-492
> 
> If it makes sense, it would be nice to use these keywords as first-level 
> features in Twisted.
> 
> Since Amber brought up discussion of dropping Python 2.7 here:
> https://twistedmatrix.com/pipermail/twisted-python/2019-March/032234.html
> 
> I thought I would raise dropping Python 3.4 also.
> --
> Craig
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
> 
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] A Proposal for reducing the burden of developing on Twisted by dropping Python 2 support

[Tom Most]

This also works for me and at $EMPLOYER. We expect the same or similar to 
happen in many more parts of the Python ecosystem.

-- 
  Tom Most
  t...@freecog.net

On Mon, Mar 25, 2019, at 3:15 AM, Amber Brown wrote:
> Hi everyone,
> 
> Since the Python 2 EOL date is rapidly approaching, I thought it was 
> time we consider dropping Python 2 support.
> 
> I personally find that Python 2 compat adds a huge amount of overhead 
> when working on and maintaining Twisted, and think that with the current 
> maintainer availability, dropping it sooner rather than later would have 
> a beneficial effect on how much work we spend on shims/compat, 
> complexity, and our ability to ship new features, as well as onboarding 
> people who are interested in the project, but have no interest (or 
> experience!) in Python 2.7.
> 
> It is basically summed up by doing a feature freeze on an agreed-upon 
> version of Twisted, that will be the last version released for 2.7. It 
> would be abnormal in that it would get security fixes (our current 
> policy is to only release them for current versions) and critical 
> bugfixes, but would otherwise be frozen.
> 
> One of my rationales is that from some analysis of PyPI download 
> statistics, the vast majority of Python 2 users are using old versions 
> of Twisted, while nearly all our Python 3 users are on the latest 
> version. As such, I believe freezing a version that will get security 
> updates but no new features would not be a massive loss to those stuck 
> on Python 2 for whatever reason.
> 
> Twisted's compatibility policy would still apply, ensuring that Python 
> 2/3 compatible software using Twisted would be able to use the older 
> Twisted version on Python 2, and the newer version on Python 3, as you 
> would usually expect.
> 
> You can find the proposal here, in this handy-dandy Google Doc: 
> https://docs.google.com/document/d/1S4CGgZC09blLIdk3Zo7wBa75A9_JuuH_3akkyjN0lik/edit
> 
> Comments are welcome, as well as which timeline seems reasonable.
> 
> - Amber
> 
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Twisted 19.2.0rc1 Release Candidate Announcement

[Tom Most]

Another note:

> twisted.web.client.Request and twisted.web.client.HTTPClient were both 
> vulnerable to header injection attacks. They now replace linear whitespace 
> ('r', 'n', and 'rn') with a single space. (#9421)

Headers now assumes the value is a list of string (or bytes). It used to accept 
a list of arbitrary values. When using the twisted.web resource model (the 
IRequest.setHeader API) the value would be coerced.

This change actually broke some of my tests, though not any production code. I 
was passing a list where a scalar string was required (this, precisely: 
https://github.com/twisted/treq/pull/237). The result was a response with a 
header like:

Foo: ['value']

That's not a useful feature in this case so I am happy to have caught the bug 
but it does seem like there may be real-world code this breaks. For example, 
request.setHeader('Retry-After', 10) would produce a valid header.

(Also the release note seems to be missing some backslashes in the bit I 
quoted.)

---Tom

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] twisted.web with dynamic + static content

[Tom Most]

On Mon, Nov 19, 2018, at 3:06 PM, Jeff Grimmett wrote:
> Glyph,
> 
> Belated (I was in the thick of a few unrelated crises) thanks - it
> helps to understand a little bit of the background on the way it
> works.  I have made adjustments and everything works the way I was
> hoping for now.> 
> I wish I understood it a little better, if for no other reason than to
> contribute some documentation tweaks.  Just how would that work,
> anyway? The contribution model for Twisted is a little opaque from
> where I am sitting. I'd love to help in areas where I am actually
> capable of doing :)> 
> Regards,
> 
> Jeff 

Hi Jeff,

The Twisted development process is documented here on the wiki[1]. It
boils down to:
1. Create a ticket[2] (on Trac)
2. Create a pull request (on GitHub)
3. Wait for a review (and address any feedback the reviewer gives)

There is more on the wiki. The same process applies to code and
documentation.
Documentation changes are really helpful, especially because it is
difficult for the experienced to see the rough edges that catch new
users. Your confusion is a bug, and particularly with respect to the
Resource model, a not uncommon one. Let's fix it!
---Tom




Links:

  1. 
https://twistedmatrix.com/trac/wiki/TwistedDevelopment#GettingYourPatchAccepted
  2. https://twistedmatrix.com/trac/newticket
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] automated tests for a server application

[Tom Most]

On Tue, Nov 6, 2018, at 8:43 AM, Kyle Altendorf wrote:
> On 2018-11-06 11:28, Chris Withers wrote:
> I guess I'm still not clear on what the point of using a 'fake' reactor 
> over a 'real' one is.  Not that I'm an expert here...

There are a bunch of advantages! Including:

1. Your tests become more deterministic --- even *fully* deterministic.
2. Fake I/O is way faster --- no syscalls!
3. Fake I/O is more reliable --- no worries about failing to allocate a port.
4. You gain full control over time. This makes it easy to test behaviors like 
timeouts and backoff timers.
5. You can run multiple fake reactors. Great for testing distributed systems!

At an extreme, you can make the whole suite synchronous (meaning, Deferreds 
fire immediately). treq.testing[1] can be used like this. [2] is an example of 
a suite in this style (this suite is actually run under the Django test runner, 
which knows nothing of reactors or Deferreds, though I don't recommend doing 
that).

There are some potential pitfalls:

1. As with any test double, you can trick yourself. I find faking I/O in this 
way is less prone to these problems than tools like unittest.mock, though.
2. Manually controlling time can harm composability. Application-level Twisted 
tends to eventually require some sort of time-related behavior, if only the 
humble reactor.callLater(0, ...) to avoid stack overflow. (Use of the 
cooperator from twisted.internet.task is another common one.)
3. Some (old) bits of Twisted don't allow passing a reactor or use the one 
passed in all code paths[3].

You can combine fake I/O with techniques like generative testing, too, though I 
don't have any public examples to point at. And of course you can also combine 
with other forms of dependency injection, like passing in an endpoint.

FWIW twisted.test.proto_helpers and twisted.test.iosim *are* public API, 
despite their presence in the test package. They could certainly use docs, and 
there is a ticket to move them somewhere more appropriate. For the moment the 
best way to learn to read them is to look at Twisted's own test suite. I found 
reading the tests for HostnameEndpoint helpful [4].

---Tom

[1]: https://treq.readthedocs.io/en/latest/testing.html
[2]: 
https://github.com/twm/yarrharr/blob/e31a488f8884fb82f1dba733585bef4a7ad86968/yarrharr/tests/test_fetch.py#L282
[3]: 
https://github.com/twm/yarrharr/pull/277/commits/1a5b5832edbf1fb1b1f45e9d99b65dad51ada566#diff-29ffe25b52ad7bddee9f2f08544e899cR98
[4]: 
https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/test/test_endpoints.py

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Can transport.write() to hostname instead of IP address?

[Tom Most]

Hi Sean,

To summarize, you are writing a client application using Twisted which
needs to talk with a service behind an Nginx reverse proxy. The reverse
proxy requires use of TLS (a.k.a. SSL) and SNI to identify the
appropriate backend service.
If you were using HTTP, Twisted's twisted.web.client.Agent[1] API
already does the right thing here -- it takes the hostname from the URL
it is given and populates SNI.
For your custom protocol on top of TLS, you should use a TLS endpoint to
connect to the server from your client. This is a little difficult to
discover because there isn't a TLS endpoint per se -- instead, there is
a function which wraps another endpoint. See the TLS section in the
endpoint documentation[2], which includes this example:
wrapped = HostnameEndpoint('example.com', 443) contextFactory =
optionsForClientTLS(hostname=u'example.com') endpoint =
wrapClientTLS(contextFactory, wrapped) conn =
endpoint.connect(Factory.forProtocol(Protocol))
I'll break this down:

1. HostnameEndpoint will resolve the hostname to an IP address and
   creates a TCP connection to port 443.2. The optionsForClientTLS[3] function 
generates an object which
   represents the TLS connection options. Importantly, it enables SNI
   based on the hostname passed to it.3. wrapClientTLS[4] returns an endpoint 
which layers TLS on top of the
   plain TCP connection generated by HostnameEndpoint. It also takes the
   TLS options as an argument.4. conn is a Deferred which will fire with a 
protocol instance generated
   by the factory passed to connect().
This is basically what Agent does internally, as I understand it.

Hope this helps,
Tom

On Thu, Aug 16, 2018, at 6:44 PM, Sean DiZazzo wrote:
> I guess thats still kind of confusing without making something more
> clear...> 
> In my example, both myprotocol.example.com and test.example.com DNS
> records would point to the same IP address.  One nginx instance then
> listens on that IP and serves up several ssl apps. They go through a
> "mapper" that uses the SNI and the ssl_preread directive to read the
> destination hostname of the packet to determine which app to route the
> traffic to.> 
> I just want transport.write() to not resolve the ip address of the
> host I pass in.  Everything will work if it connects and sends packets
> to myprotocol.example.com:443[5] instead of 23.23.23.23:443.> 
> Nginx reference:
> http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html
> 
> On Thu, Aug 16, 2018 at 6:14 PM, Sean DiZazzo
>  wrote:>> Thanks for responding, Adi!
>> 
>> I don't want each packet to go it's own way from Twisted.  They all
>> go to the same place from each instance of the server/protocol.  They
>> go to my custom protocol listening on another local port.>> 
>> It's just that I'm serving up several different ssl apps on the same
>> nginx server, and nginx uses the hostname to route the packets.  So
>> in this case, traffic coming in on http.example.com:443 might be
>> routed to an https app listening on a socket, and traffic coming in
>> to myprotocol.example.com:443 should be routed to my own protocol
>> listening on port .  So if nginx doesn't get the hostname, it
>> doesn't know to route the packet to my custom protocol instead of the
>> web server.  Does that make sense?>> 
>> It seems that the transport is resolving the hostname to an ip
>> address and then sending the traffic to the generic ip which is not
>> enough info for nginx to route the packet correctly.>> 
>> On Thu, Aug 16, 2018 at 5:49 PM, Adi Roiban  wrote:
>>> On Fri, 17 Aug 2018 at 01:25, Sean DiZazzo 
>>> wrote:
>>>  >
>>>  > Hi all!
>>>  >
>>>  > After I start a reactor connecting to a specific hostname and
>>>  > port, I do my thing and then call transport.write() to send the
>>>  > data to the peer.
>>>  >
>>>  > From what I can tell, though, the hostname is resolved, and the
>>>  > data is written back to the ip address itself, instead of the
>>>  > hostname I started the reactor with.
>>>  >
>>>  > This is a problem in my case because we are using nginx's
>>>  > ssl_preread server_name directive to route several different
>>>  > streams all coming in on the same ip address.
>>>  >
>>>  > So the write() method needs to explicitly use the hostname to
>>>  > route the packet properly.
>>>  >
>>>  > So... Is there any way to have transport.write() use the hostname
>>>  > given instead of it's resolved IP address? Or am I missing
>>>  > something?
>>>  >
>>>
>>> I assume you are using TCP here.>>> 
>>>  I guess that you are missing something.
>>> 
>>>  If you want each write to go over its own way / route and have the>>>  
>>> hostname re-resolved you should open + write + close a
>>>  connection for>>>  each write.
>>> 
>>>  But I think that there is something else there and this is now what
>>>  you want :)>>>  Do you use HTTP or have a custom protocol?
>>> 
>>>  Cheers,
>>> 
>>>  Adi Roiban
>>> 
>>>  ___
>>>  Twisted-Python mailing list

Re: [Twisted-Python] "Cleared to Land"?

[Tom Most]

On Sun, Jul 8, 2018, at 11:45 PM, Glyph wrote:
>> On Jul 8, 2018, at 6:21 PM, Tom Most  wrote:
>> 
>> On Sun, Jul 8, 2018, at 12:06 PM, Glyph wrote:
>>>> On Jul 8, 2018, at 7:44 AM, Adi Roiban  wrote:
>>>> 
>>>> On Sat, 7 Jul 2018 at 22:45, Glyph  wrote:>>>>> 
>>>> On Jul 7, 2018, at 12:36 PM, Tom Most  wrote:
>>>> 
>>>> [snip]
>>>> 
>>>>> Thanks for driving this to completion, Tom!  Much appreciated.
>>>>> 
>>>>> For future reference though, when landing stuff on trunk, please
>>>>> use this format exactly:>>>>> 
>>>>> https://twistedmatrix.com/trac/wiki/ReviewProcess#Authors:Howtomergethechangetotrunk>>>>>
>>>>>  
>>>>> This helps with both automatically closing the associated Trac
>>>>> ticket (I already did that in this case) and properly attributing
>>>>> credit for reviewer points on
>>>>> https://twistedmatrix.com/highscores/.>>>> 
>>>> I always forget the format and then I have to search previous
>>>> commits or wiki.>>>> 
>>>> We make reference to the Trac ticket in PR title, Branch Name, PR
>>>> description (As part of checklist).
>>>> 
>>>> Do we really need to make another reference in the merge commit?
>>>> For me, it's red tape.
>>>> 
>>>> I don't think that we should make things more complicated, just to>>>> 
>>>> have a functional https://twistedmatrix.com/highscores/
>>> 
>>> If we forget every so often, it's not a big deal, but I find it's
>>> nice to have a generally consistent merge format.  I do really wish
>>> that Github could do this in the same way that it does for a PR
>>> template so we didn't have to remember manually.>> 
>> Ah, sorry. It looks like I've done this a few times. I had wondered
>> why my points weren't changing.>> 
>> A template would be nice, if only as a reminder, but I think that I
>> will always struggle with this if I use the GitHub UI to merge --- no
>> other project to which I contribute has requirements w.r.t. the merge
>> commit message, so I am fighting habit. I'll look into scripting it.>> 
>> As an aside, could you update "" in the merge
>> commit message example to ""? I
>> always have to look at the repository history to see if this is
>> supposed to be full names or Trac/GitHub handles.> 
> I've just made you a trac admin, so you should be able to have
> at it :-).
Thanks, done! :-)

>>>>> Perhaps we should avoid the "cleared to land" label on PRs from
>>>>> non-committers? I scan through open PRs for ones which require a
>>>>> procedural nudge now and again, but I had not looked at this one
>>>>> as the process appeared to be done with it, and I missed that it
>>>>> was from a non-committer.>>>>> 
>>>>> 
>>>>> Ideally it would be used sparingly, but, the availability of such
>>>>> a process release-valve allows someone to do a review even if they
>>>>> only have time to read the code, and not the time to sit around
>>>>> waiting 40 minutes for some intermittent CI nonsense to shake out.>>>>> 
>>>>> Since this doesn't have a https://twisted.reviews/ -like "core
>>>>> gameplay loop" that project members regularly visit, perhaps if
>>>>> you're going to use this label you should always shoot a message
>>>>> to this list as well, to let other contributors know that they
>>>>> should take a look if they have a minute?>>>>> 
>>>>> On that note, it looks like
>>>>> https://github.com/twisted/twisted/pull/1010 has suffered the same
>>>>> fate?  Any other committers have a minute to land that one? :)>>>> 
>>>> https://github.com/twisted/twisted/pull/1010 was merged now.
>>>> 
>>>> Do we really need and up to date branch before a merge?
>> 
>> The GitHub UI currently requires this for non-admins, though it can
>> be bypassed with a manual merge, much like the requirement for a
>> green build. As a new committer I haven't felt comfortable doing that
>> bypass, so I spend a lot of time merging forward and kicking builds
>> to get a green check mark. Should I be bothering? Particularly for
>> documentation changes[1]. I'd rather spend tha

Re: [Twisted-Python] "Cleared to Land"?

[Tom Most]

On Sun, Jul 8, 2018, at 6:21 PM, Tom Most wrote:
> The GitHub UI currently requires this for non-admins, though it can be
> bypassed with a manual merge, much like the requirement for a green
> build. As a new committer I haven't felt comfortable doing that
> bypass, so I spend a lot of time merging forward and kicking builds to
> get a green check mark. Should I be bothering? Particularly for
> documentation changes[1]. I'd rather spend that time on work product
> than process.
Please disregard this. I tried it on #1037[2] but branch protection
still applies:
$ git push origin trunk
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 610 bytes | 0 bytes/s, done.
Total 6 (delta 4), reused 0 (delta 0)
remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
remote: error: GH006: Protected branch update failed for 
refs/heads/trunk.remote: error: 3 of 3 required status checks are expected.
To g...@github.com:twisted/twisted.git
! [remote rejected] trunk -> trunk (protected branch hook declined)
error: failed to push some refs to 'g...@github.com:twisted/twisted.git'
---Tom

Links:

  1. https://github.com/twisted/twisted/pull/1037
  2. https://github.com/twisted/twisted/pull/1037
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] "Cleared to Land"?

[Tom Most]

On Sun, Jul 8, 2018, at 12:06 PM, Glyph wrote:
>> On Jul 8, 2018, at 7:44 AM, Adi Roiban  wrote:
>> 
>> On Sat, 7 Jul 2018 at 22:45, Glyph  wrote:
>>> On Jul 7, 2018, at 12:36 PM, Tom Most  wrote:
>> 
>> [snip]
>> 
>>> Thanks for driving this to completion, Tom!  Much appreciated.
>>> 
>>> For future reference though, when landing stuff on trunk, please use
>>> this format exactly:>>> 
>>> https://twistedmatrix.com/trac/wiki/ReviewProcess#Authors:Howtomergethechangetotrunk>>>
>>>  
>>> This helps with both automatically closing the associated Trac
>>> ticket (I already did that in this case) and properly attributing
>>> credit for reviewer points on https://twistedmatrix.com/highscores/.>> 
>> I always forget the format and then I have to search previous commits
>> or wiki.>> 
>> We make reference to the Trac ticket in PR title, Branch Name, PR
>> description (As part of checklist).
>> 
>> Do we really need to make another reference in the merge commit?
>> For me, it's red tape.
>> 
>> I don't think that we should make things more complicated, just to
>> have a functional https://twistedmatrix.com/highscores/
> 
> If we forget every so often, it's not a big deal, but I find it's nice
> to have a generally consistent merge format.  I do really wish that
> Github could do this in the same way that it does for a PR template so
> we didn't have to remember manually.
Ah, sorry. It looks like I've done this a few times. I had wondered why
my points weren't changing.
A template would be nice, if only as a reminder, but I think that I will
always struggle with this if I use the GitHub UI to merge --- no other
project to which I contribute has requirements w.r.t. the merge commit
message, so I am fighting habit. I'll look into scripting it.
As an aside, could you update "" in the merge
commit message example to ""? I always
have to look at the repository history to see if this is supposed to be
full names or Trac/GitHub handles.
>>> Perhaps we should avoid the "cleared to land" label on PRs from non-
>>> committers? I scan through open PRs for ones which require a
>>> procedural nudge now and again, but I had not looked at this one as
>>> the process appeared to be done with it, and I missed that it was
>>> from a non-committer.>>> 
>>> 
>>> Ideally it would be used sparingly, but, the availability of such a
>>> process release-valve allows someone to do a review even if they
>>> only have time to read the code, and not the time to sit around
>>> waiting 40 minutes for some intermittent CI nonsense to shake out.>>> 
>>> Since this doesn't have a https://twisted.reviews/ -like "core
>>> gameplay loop" that project members regularly visit, perhaps if
>>> you're going to use this label you should always shoot a message to
>>> this list as well, to let other contributors know that they should
>>> take a look if they have a minute?>>> 
>>> On that note, it looks like
>>> https://github.com/twisted/twisted/pull/1010 has suffered the same
>>> fate?  Any other committers have a minute to land that one? :)>> 
>> https://github.com/twisted/twisted/pull/1010 was merged now.
>> 
>> Do we really need and up to date branch before a merge?

The GitHub UI currently requires this for non-admins, though it can be
bypassed with a manual merge, much like the requirement for a green
build. As a new committer I haven't felt comfortable doing that bypass,
so I spend a lot of time merging forward and kicking builds to get a
green check mark. Should I be bothering? Particularly for documentation
changes[1]. I'd rather spend that time on work product than process.
>> I was thinking that with the "clear to land" label, we can have a
>> robot which checks for the PR and once all tests are green, the robot>> will 
>> automatically merge based on
>> https://developer.github.com/v3/pulls/#merge-a-pull-request-merge-button> 
> This sounds like a great idea, and is very close to what I would do if
> I had time for working on automation :).> 
> Rather than using the "clear to land" label, however, I'd probably do
> this on the basis of a passing review from a project member being
> present, and a passing build.  (The "clear to land" convention
> predates Github native code reviews.)
I think we'd still need a label or other explicit indicator that an
automatic merge should occur, or we won't be able to leave a passing
review which suggests trivial fixes. "Please fix this typo, then me

Re: [Twisted-Python] "Cleared to Land"?

[Tom Most]

On Sat, Jul 7, 2018, at 12:08 AM, Glyph wrote:
> 
>> 
>> On Jul 6, 2018, at 4:11 PM, Wim Lewis  wrote:
>> 
>> I have a minor PR that was reviewed and labeled "cleared to land" a
>> while back, but three months have passed since then and it's still
>> unmerged. Is there something else I need to do to move this along, or>> did 
>> it just fall through the cracks?
>> 
>> Wim.
> 
> 
> Hi Wim,
> 
> Can you include a link to to the issue in question?  Probably
> someone (me?) intended to indicate that it was in the appropriate
> workflow state for landing, but was waiting for continuous
> integration to complete, and wandered off and forgot about it.  If
> you can pester a contributor here, or on IRC, someone should be able
> to land it for you.> 
> Speaking of which, it seems like you've been a fairly stalwart and
> patient external contributor.  Would you like to join the project as a
> member and merge your own approved stuff to trunk?  Mostly, the
> attendant responsibilities are documented here:> 
> https://twistedmatrix.com/trac/wiki/ReviewProcess
> 
> (This used to be the authoritative list -
> https://twistedmatrix.com/trac/wiki/CommitterCheckList - and I've just
> updated it to remove some extraneous stuff, but it seems that
> pervasive cultural understanding of github's workflow have obviated
> the need for repetitive stern cautions that one should not commit
> directly to trunk without code review and CI...)> 
> -g

Hi all,

Perhaps it was https://github.com/twisted/twisted/pull/991? It was
tagged "cleared to land", but had build failures unrelated to the
changes. I merged forward, pushed to branch, kicked a failing build, and
got the green checkmark. It has been merged.
Perhaps we should avoid the "cleared to land" label on PRs from non-
committers? I scan through open PRs for ones which require a procedural
nudge now and again, but I had not looked at this one as the process
appeared to be done with it, and I missed that it was from a non-
committer.
Also, yes, welcome Wim!

---Tom


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Removing support for old Python 3 versions

[Tom Most]

Yes, this is what my Raspbian system reports:

Python 3.4.2 (default, Oct 19 2014, 13:31:11)
[GCC 4.9.1] on linux

---Tom

On Fri, Feb 24, 2017 at 6:01 PM, Glyph Lefkowitz <gl...@twistedmatrix.com>
wrote:

>
> On Feb 24, 2017, at 3:29 PM, Tom Most <tomm...@gmail.com> wrote:
>
> On Thu, Feb 23, 2017 at 1:30 AM, Glyph Lefkowitz <gl...@twistedmatrix.com>
> wrote:
>
>>
>> Is there a "pre-installed on hardware that's hard to change" issue here?
>> Do Raspbian users generally upgrade to new stable releases?
>>
>> -g
>>
>
> There's also a substantial lag after a Debian release before the
> corresponding version of Raspbian becomes generally available. Debian
> Jessie was released April 2015, but Raspbian Jessie looks to have gone GA
> in September[1] (though you could get it sooner by adjusting your apt
> sources).
>
> [1]: https://www.raspberrypi.org/blog/raspbian-jessie-is-here/
>
>
> So Jesse (== CPython 3.4) is still what's current for Raspbian users?
>
> -glyph
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] treq POST abborting with: err: ('Could not adapt', '{"....", "..."} )

[Tom Most]

Sure, I'd be happy to help (but won't get to it until the weekend).
Username is twm.

Thanks,
Tom

On 01/11/2017 04:44 PM, Glyph Lefkowitz wrote:
>
>> On Jan 10, 2017, at 6:43 PM, Tom Most <tomm...@gmail.com
>> <mailto:tomm...@gmail.com>> wrote:
>>
>> On 01/08/2017 06:49 PM, Glyph Lefkowitz wrote:
>>>
>>>> On Jan 6, 2017, at 11:03 AM, steven meiers <commercial...@yahoo.de>
>>>> wrote:
>>>>
>>>> hi,
>>>>
>>>>
>>>> sending a simple post request with a body (cookies) results in a error
>>>> here.
>>>>
>>>> im using python 3.x with the latest twisted and treq installed via pip
>>>> in a virtualenv.
>>>
>>> Something that we discussed on IRC but did not come up in the
>>> mailing list thread may be the one issue that *should* be fixed in treq:
>>>
>>> Because of this issue, this example
>>> - http://treq.readthedocs.io/en/latest/#post - which the OP was
>>> following, doesn't work on Python 3.
>>>
>>> We should fix the docs so that it does.
>>
>> I submitted a PR to fix this a while ago:
>> https://github.com/twisted/treq/pull/152
>>
>> Also, the docs on RTD are still out of date. Who owns them? I'd be
>> happy to help.
>
> Thanks, hopefully the next time my periodic review comes around I'll
> catch that one.
>
> Looks like I own the docs.  Builds are apparently failing - should I
> add you to the project?
>
> -glyph
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] treq POST abborting with: err: ('Could not adapt', '{"....", "..."} )

[Tom Most]

On 01/08/2017 06:49 PM, Glyph Lefkowitz wrote:
>
>> On Jan 6, 2017, at 11:03 AM, steven meiers > > wrote:
>>
>> hi,
>>
>>
>> sending a simple post request with a body (cookies) results in a error
>> here.
>>
>> im using python 3.x with the latest twisted and treq installed via pip
>> in a virtualenv.
>
> Something that we discussed on IRC but did not come up in the mailing
> list thread may be the one issue that *should* be fixed in treq:
>
> Because of this issue, this example
> - http://treq.readthedocs.io/en/latest/#post - which the OP was
> following, doesn't work on Python 3.
>
> We should fix the docs so that it does.

I submitted a PR to fix this a while ago:
https://github.com/twisted/treq/pull/152

Also, the docs on RTD are still out of date. Who owns them? I'd be happy
to help.

---Tom
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Exception's Implicit Public API, and Python 3

[Tom Most]

Would str(failure.value) work?  It should be stable from 2.x to 3.x,
barring Unicode differences:

https://docs.python.org/2/library/exceptions.html#exceptions.BaseException
https://docs.python.org/3.5/library/exceptions.html#BaseException

(This convention is quite handy when you want to defer formatting of a
fancy exception message.  Just pass args to the constructor and override
__str__ to do the expensive formatting.)

---Tom

On 09/06/2015 01:20 PM, Tobias Oberstein wrote:
>> Put differently, Twisted's "the first one's free" policy applies to
>> upgrading Twisted itself, and /not/ to upgrading Python (or any other
>> dependency).  If you upgrade Python and you need to update your code for
>> that, Twisted won't create any additional problems but it won't go out
>> of its way to solve the ones you normally have to deal with.
>>
>> Sound good?
>
> Sounds good. We're fine.
>
> FWIW, we need to guard for even more cases
>
> .message there
> .args there _and_ a tuple _and_ of length > 0
>
> https://github.com/tavendo/AutobahnPython/commit/303d289de4993b5ffa9bf90c6fed71364f3521e7#diff-ced6a7641cc1e78205bd85d77b484c03R111
>
>
> /Tobias
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python