[Twisted-Python] we can stop depending on pypiwin32

[Glyph]

As per the release announcement in 
https://mail.python.org/pipermail/python-win32/2018-February/014000.html 
, 
PyWin32 now does development on Github, and uploads binary wheels to PyPI, as 
you can see here: https://pypi.org/project/pywin32/223/#files 
.  So we can stop depending on my 
ill-fated pypiwin32 fork.

If anybody is interested in doing some minor Windows maintenance, I would like 
very much to never think about PyPIWin32 again :).

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] RunningTwisted tests on Windows and SSL

[Craig Rodrigues]

Do not run the tests directly from the source tree.
Instead, you should install Twisted into a virtualenv
and run "trial twisted" from inside the virtualenv and outside the source
tree.
The installer has some logic to omit the installation of certain modules
and tests
which do not work on Python 3.

You can also run the tests by using tox as specified here:

https://github.com/twisted/twisted/blob/trunk/README.rst#unit-tests

because tox takes care of setting up the virtualenv and running the
tests inside the virtualenv.

--
Craig



On Sat, Mar 17, 2018 at 2:58 AM, John Aherne  wrote:

>
>
> twisted.cred.test.test_strcred.SSHCheckerTests.test_isChecker
> 
> ===
> [ERROR]
> Traceback (most recent call last):
>   File "c:\twistedssl\twisted\src\twisted\mail\test\test_mail.py", line
> 12, in 
> import StringIO
> builtins.ModuleNotFoundError: No module named 'StringIO'
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Strategy for finding source of unhandled error in Deferred

[Jean-Paul Calderone]

On Sun, Mar 18, 2018 at 11:21 AM, Jean-Paul Calderone <
exar...@twistedmatrix.com> wrote:

> On Sun, Mar 18, 2018 at 11:04 AM, Alex Railean  wrote:
>
>> Hello,
>>
>> I have inherited a misbehaving system and am trying to troubleshoot it.
>> The code
>> is not documented and I cannot produce a minimal working example that
>> illustrates
>> the problem yet, as I don't know which parts are involved and how they
>> relate.
>>
> This produces the following entries in the log:
>> Unhandled error in Deferred:
>> CRITICAL _legacy   twcore Unhandled error in Deferred:
>>
>> Traceback (most recent call last):
>> Failure: __builtin__.type: 
>> CRITICAL _legacy   twcore
>>
>>
>>
>> How can I get more clues about what is going on? I was expecting a line
>> number,
>> or a complete call-stack.
>>
>
Try this:

from twisted.internet.defer import setDebugging
setDebugging(True)

This gives you creation and callback stacks in the logs.

Jean-Paul


>
>>
>> What are the recommended troubleshooting steps for finding the Deferred
>> in which
>> the error occurred?  There are are around 30 of them throughout the code,
>> is
>> there a heuristic I can use to narrow down the list of suspects?
>>
>>
>>
>>
>>
>>
>>
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Strategy for finding source of unhandled error in Deferred

[Jean-Paul Calderone]

On Sun, Mar 18, 2018 at 11:04 AM, Alex Railean  wrote:

> Hello,
>
> I have inherited a misbehaving system and am trying to troubleshoot it.
> The code
> is not documented and I cannot produce a minimal working example that
> illustrates
> the problem yet, as I don't know which parts are involved and how they
> relate.
>
This produces the following entries in the log:
> Unhandled error in Deferred:
> CRITICAL _legacy   twcore Unhandled error in Deferred:
>
> Traceback (most recent call last):
> Failure: __builtin__.type: 
> CRITICAL _legacy   twcore
>
>
>
> How can I get more clues about what is going on? I was expecting a line
> number,
> or a complete call-stack.
>
>
> What are the recommended troubleshooting steps for finding the Deferred in
> which
> the error occurred?  There are are around 30 of them throughout the code,
> is
> there a heuristic I can use to narrow down the list of suspects?
>
>
>
>
>
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Strategy for finding source of unhandled error in Deferred

[Alex Railean]

Hello,

I have inherited a misbehaving system and am trying to troubleshoot it. The code
is not documented and I cannot produce a minimal working example that 
illustrates
the problem yet, as I don't know which parts are involved and how they relate.

The version of Twisted used is 16.1.1, running on Fedora 15.


My first attempt is to leverage Twisted's logging capabilities and retrieve
information about the place in the code that leads to the error:

from twisted.python import log as twisted_log
observer = twisted_log.PythonLoggingObserver(loggerName='twcore')
observer.start()



This produces the following entries in the log:
Unhandled error in Deferred:
CRITICAL _legacy   twcore Unhandled error in Deferred:

Traceback (most recent call last):
Failure: __builtin__.type: 
CRITICAL _legacy   twcore



How can I get more clues about what is going on? I was expecting a line number,
or a complete call-stack.


What are the recommended troubleshooting steps for finding the Deferred in which
the error occurred?  There are are around 30 of them throughout the code, is
there a heuristic I can use to narrow down the list of suspects?








___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] RunningTwisted tests on Windows and SSL

[John Aherne]

This is useful info for anyone else running into these issues. At least
they might save a bit of time not having to run down what is happening.


Seems that the mail,email, test_hook and StringIO errors all come from not
ported modules as seen in twisted\src\python\_setup.py.

The _getEllipticCurve error

<< builtins.AttributeError: type object 'OpenSSLCertificateOptions' has no
attribute '_getEllipticCurve'

twisted.test.test_sslverify.OpenSSLOptionsTests.test_ecDoesNotBreakConstructor
>>

is because getattr will raise an AttributeError, which is what it is doing
here. But it does not do this for python2.7.

The test is not to break if the ECCurve is missing. It is missing since it
is not found. Still looking at this.

With wincertstore being used, we get a failing test as follows:

[FAIL]
Traceback (most recent call last):
  File "c:\twistedssl\twisted\src\twisted\web\test\test_agent.py", line
1568, in test_setsTrustRootOnContextToDefaultTrustRoot
certificateOptions.trustRoot, ssl.OpenSSLDefaultPaths)
  File "c:\twistedssl\twisted\src\twisted\trial\_synctest.py", line 649, in
assertIsInstance
instance, classOrTuple, suffix))
twisted.trial.unittest.FailTest:
 is not an instance of 

twisted.web.test.test_agent.WebClientContextFactoryTests.test_setsTrustRootOnContextToDefaultTrustRoot

So a new test is needed to fix this.

Also, the current patch only loads the ROOT certificates. The standard
library loads ['CA', 'ROOT'], so we should be doing the same I imagine.



-- 
*John Aherne*




*www.rocs.co.uk *
020 7223 7567
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Buildmaster is down

[Adi Roiban]

On 18 March 2018 at 04:56, Glyph  wrote:
>
>
>> On Mar 17, 2018, at 11:16 AM, Adi Roiban  wrote:
>>
>> Hi,
>>
>> Today, I tried to reconfigure the Buildbot master from braid/master
>> and it failed.

[snip]

> I don't believe the issue was changes made outside Braid; based on the 
> failure mode it looks to me like you had an out-of-date or edited 
> twisted-infra-secret repo.  Please be sure to git fetch (git reset --hard 
> origin/master ; git secret reveal) before doing anything in prod.

Thanks. `git secret reveal` was the issue on my side.
I knew about git secret, but I was expecting that there will be a
warning when the decrypted version is out of sync at pull.

There is `fab config.production buildbot.reconfigure` which should
just touch the configuration, without updating the virtualenv.
Usually this is much faster.

Thanks again for the fix, my bad for getting into this!
Adi

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] New member of the twisted/twisted-contributors team.

[Glyph]

> On Mar 18, 2018, at 12:50 AM, Ilya Skriblovsky  
> wrote:
> 
> Thanks, I greatly appreciate!
> 
> I'll try to find time to commit it to reviews

✨❤️✨

-g

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] New member of the twisted/twisted-contributors team.

[Ilya Skriblovsky]

Thanks, I greatly appreciate!

I'll try to find time to commit it to reviews

вс, 18 мар. 2018 г., 7:26 Glyph :

>
>
> On Mar 15, 2018, at 3:35 AM, Adi Roiban  wrote:
>
> Hi,
>
> Just a quick announcement.
>
> I made Ilya Skriblovsky (https://github.com/IlyaSkriblovsky) a member
> of twisted/twisted-contributors team.
>
> The main reason is to make it easier to work with Ilya and allow him
> to run tests on Buildbot.
>
> Let me know if you want this change reverted.
>
>
> Far from it.  We need all the help we can get right now :).
>
> Congratulations, Ilya!  I hope Adi already told you this, but please read
> https://twistedmatrix.com/trac/wiki/CommitterCheckList, and welcome to
> the team :).
>
> In the past we've had some hand-wringing on this list about how exactly
> our "become a committer" process should work.  Given the growing length of
> the review queue, I should just say: if you want to become a Twisted
> project member, now's your chance.  If you'll commit to doing even 1 review
> a month, I'll be happy to hand out repo:write privileges at this point to
> anyone whose name I can passably recognize and who has a stated interest :).
>
> -g
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] RunningTwisted tests on Windows and SSL

[John Aherne]

 Some clarification on running the tests.

For the tests on test_sslverify

If I apply the patches from 6371 on python2.7.13 32bit windows2012R2, the
test_sslverify runs without error or failure.

  TrustRootTests
test_caCertsPlatformDefaults ...
 [OK]
test_caCertsWindows ...
[OK]
test_trustRootPlatformRejectsUntrustedCA ...
 [OK]
test_trustRootSpecificCertificate ...
[OK]


===
[SKIPPED]
NPN/ALPN is present on this platform

twisted.test.test_sslverify.NPNAndALPNAbsentTests.test_
NPNAndALPNNotImplemented
twisted.test.test_sslverify.NPNAndALPNAbsentTests.test_
NegotiatedProtocolReturns
None
twisted.test.test_sslverify.NPNAndALPNAbsentTests.test_
nextProtocolMechanismsNoN
egotiationSupported

---
Ran 113 tests in 496.749s

PASSED (skips=3, successes=110)

---



If I apply the patch on python3.6.3 32bit Windows10Pro, the test_sslverify
produces 1 error and 1 failure.

The test_caCertsWindows passes

Looking at the Error on  test_ecDoesNotBreakConstructor, I don't see where
I can find  _getEllipticCurve.

I can't see it in  OpenSSLCertificateOptions. And so far google does not
help


===
[FAIL]
Traceback (most recent call last):
  File "c:\twistedssl\twisted\src\twisted\test\test_sslverify.py", line
1292, in test_ecSuccessWithRealBindings
self.assertEqual(defaultCurve, opts._ecCurve)
  File "c:\twistedssl\twisted\src\twisted\trial\_synctest.py", line 432, in
assertEqual
super(_Assertions, self).assertEqual(first, second, msg)
  File "c:\program files (x86)\python36-32\Lib\unittest\case.py", line 829,
in assertEqual
assertion_func(first, second, msg=msg)
  File "c:\program files (x86)\python36-32\Lib\unittest\case.py", line 822,
in _baseAssertEqual
raise self.failureException(msg)
twisted.trial.unittest.FailTest:  !=


twisted.test.test_sslverify.OpenSSLOptionsTests.test_
ecSuccessWithRealBindings

===
[ERROR]
Traceback (most recent call last):
  File "c:\twistedssl\twisted\src\twisted\test\test_sslverify.py", line
1254, in test_ecDoesNotBreakConstructor
sslverify.OpenSSLCertificateOptions, "_getEllipticCurve", missing)
  File "c:\twistedssl\twisted\src\twisted\trial\_synctest.py", line 1086,
in patch
monkeyPatch.patch()
  File "c:\twistedssl\twisted\src\twisted\python\monkey.py", line 53, in
patch
self._originals.append((obj, name, getattr(obj, name)))
builtins.AttributeError: type object 'OpenSSLCertificateOptions' has no
attribute '_getEllipticCurve'

twisted.test.test_sslverify.OpenSSLOptionsTests.test_
ecDoesNotBreakConstructor

---
Ran 113 tests in 185.166s

FAILED (skips=3, failures=1, errors=1, successes=108)

The following is what monkeypatch is working with

test_ecDoesNotBreakConstructor ... ***PATCH*** 

***PATCH*** _getEllipticCurve
   [ERROR]


test_caCertsPlatformDefaults ...
 [OK]
test_caCertsWindows ...
[OK]
test_trustRootPlatformRejectsUntrustedCA ...
 [OK]

Any pointers would be welcome

John Aherne



-- 
*John Aherne*




*www.rocs.co.uk *
020 7223 7567
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Weekly Bug Summary

[trac]

Bug summary
__
Summary for 2018-03-11 through 2018-03-18
  Opened Closed  Total Change
Enhancements:  2  0   1383 +2
Defects:   2  2911 +0
Tasks: 0  0 93 +0
Regressions:   0  0  6 +0
Total: 4  2   2394 +2

|== Type Changes   |== Priority Changes   |== Component Changes   
|Defect:   +0  |Normal:  +3   |Conch:  -1 
|Enhancement:  +2  |Lowest:  -1   |Core:   +2 
  |Web:+1 



Total Tickets
Open Tickets



New / Reopened Bugs
__
= Normal =
[#9387] twisted.web.http.Request.addCookie should support SameSite (opened by alex)
enhancement webhttp://twistedmatrix.com/trac/ticket/9387

[#9388] Testing RSA keys should be upgraded to be compatible with OpenSSH 7.6 (opened by IlyaSkriblovsky) (CLOSED, fixed)
defect  core   http://twistedmatrix.com/trac/ticket/9388

[#9389] Add a script to run pycodestyle as an external tool (opened by adiroiban)
enhancement core   http://twistedmatrix.com/trac/ticket/9389

[#9390] AsyncioSelectorReactor should use get_event_loop. (opened by markrwilliams)
defect  core   http://twistedmatrix.com/trac/ticket/9390



Closed Bugs
__
= Normal =
[#9388] Testing RSA keys should be upgraded to be compatible with OpenSSH 7.6 (opened by IlyaSkriblovsky, closed by adiroiban, fixed)
defect  core   http://twistedmatrix.com/trac/ticket/9388

= Lowest =
[#9362] Remove "terrible hack" in t.c.ssh.filetransfer.FileTransferClient (opened by wiml, closed by Adi Roiban , fixed)
defect  conch  http://twistedmatrix.com/trac/ticket/9362



Ticket Lifetime Stats
__
Oldest open ticket - [#50] conch command-line client doesn't work in win32 (since 2003-07-12 14:41:06).
Newest open ticket - [#9390] AsyncioSelectorReactor should use get_event_loop. (since 2018-03-17 17:54:57.104236).

Mean open ticket age: 2214 days, 19:29:43.357864.
Median: 2062 days, 14:40:44.212871.
Standard deviation: 1284 days, 8:25:15.115290.
Interquartile range: 1849 days, 6:10:01.183422.

Mean time between ticket creation and ticket resolution: 635 days, 1:40:36.517985.
Median: 67 days, 6:49:20.
Standard deviation is 1030 days, 14:18:54.254817.
The interquartile range is 869 days, 21:07:09.651773.

Mean time spent in review: 104 days, 7:07:59.593714.
Median: 4 days, 16:48:39.
Standard deviation: 508 days, 7:08:07.578552.
Interquartile range: 23 days, 2:24:06.11.

Mean number of times a ticket is reviewed: 1.84005869406.
Median: 1
Standard deviation: 1.44782904633.
Interquartile range: 1.


Contributor Stats
__
In the last 4 weeks,
7 unique ticket reporters
6 unique ticket reviewers
5 unique ticket resolvers
In the last 24 weeks,
36 unique ticket reporters
11 unique ticket reviewers
12 unique ticket resolvers
In the last 48 weeks,
74 unique ticket reporters
22 unique ticket reviewers
24 unique ticket resolvers





___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python