Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
--+
 Reporter:  cjerdonek |Owner:  nobody
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Other)  |  Version:  master
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+
Changes (by timgraham):

 * component:  Uncategorized => Core (Other)
 * type:  Uncategorized => Cleanup/optimization
 * stage:  Unreviewed => Accepted


Comment:

 I wonder what's special about your system compared to other Mac OS
 contributors who didn't encounter the problem.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.8c36ff6bfe5ca0cc25e9705debb603d4%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  new
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by cjerdonek):

 * status:  closed => new
 * resolution:  worksforme =>


Comment:

 I was able to get things working by causing Mac OS X's auto-detection of
 proxy information to be skipped. This possibility is mentioned
 
[https://docs.python.org/3/library/urllib.request.html#urllib.request.ProxyHandler
 here] in CPython's `urllib.request.ProxyHandler` documentation.

 More detailed instructions appear in `urllib.request`'s module docstring.

 To disable the auto-detection, I followed the instructions in the above-
 mentioned module docstring and modified `LiveServerBase.urlopen()` to look
 like the following (this is just a proof of concept):

 {{{#!python
 def urlopen(self, url):
 import urllib.request
 # Pass an empty dictionary to disable auto-detecting the proxy.
 proxy_support = urllib.request.ProxyHandler({})
 opener = urllib.request.build_opener(proxy_support)
 urllib.request.install_opener(opener)

 return urlopen(self.live_server_url + url)
 }}}

 After doing this, the following command worked fine:

 {{{
 $ ./runtests.py servers.tests
 }}}

 Since this seems like a fundamental bug / limitation on Mac OS X (in at
 least some versions), it seems worth considering including Mac-specific
 code to enable the parallel running of tests on this platform.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.cdb14ead49879e5e5208bf1622e67898%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  closed
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:  worksforme
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by cjerdonek):

 I narrowed this down further to the following:

 The hang happens when calling `urllib.request.urlopen()` inside
 `LiveServerBase`.  And digging further, inside CPython's request.py, the
 hang happens when calling `_scproxy._get_proxies()`.

 There seem to be known issues around `_scproxy` in CPython for Mac OS X
 users.  For example, see the following two issues in CPython's bug
 tracker:

 (1) Apple-supplied libsqlite3 on OS X is not fork safe;  can cause
 crashes:

 http://bugs.python.org/issue27126

 (See this comment, in particular:
 http://bugs.python.org/issue27126#msg266405 )

 (2) exception error in _scproxy.so when called after fork

 http://bugs.python.org/issue13829

 There is a chance that there are easy workarounds on the Django side that
 can be made to accommodate Mac OS X users.

 Also, for the record I'm using 10.11.6.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.c0a84a6ed43f2b3202c74de85586f246%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  closed
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:  worksforme
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by timgraham):

 * status:  new => closed
 * resolution:   => worksforme


Comment:

 Feel free to reopen if you find that Django is at fault.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.eca6e135c80ace35d44a289027a5a6af%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  new
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by bpeschier):

 Just ran the tests at the mentioned commit on my macOS Sierra public beta
 with a fresh 3.5.2 python environment. No problems there.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.0f6dff6a64483800d2d175a6ed83a167%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  new
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by timgraham):

 You could check if the issue is present at the time that parallel test
 running was added in Django and if not, bisect to find the commit that
 introduced the problem.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.73aa0027b44e18347f50505e444e7ed7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  new
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--

Comment (by cjerdonek):

 8, but it also happens when I pass `--parallel 2`. To narrow it down, it
 happens when I run the following (I believe the test setup executes, but
 it never gets to running the tests):

 {{{
 $ ./runtests.py servers.tests.LiveServerDatabase
 servers.tests.LiveServerViews
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.1f0d1ed9520f3047fa7adee21cbdaf4a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #27086: running servers.tests hangs in parallel mode

[Django]

#27086: running servers.tests hangs in parallel mode
---+--
 Reporter:  cjerdonek  |Owner:  nobody
 Type:  Uncategorized  |   Status:  new
Component:  Uncategorized  |  Version:  master
 Severity:  Normal |   Resolution:
 Keywords: | Triage Stage:  Unreviewed
Has patch:  0  |  Needs documentation:  0
  Needs tests:  0  |  Patch needs improvement:  0
Easy pickings:  0  |UI/UX:  0
---+--
Changes (by timgraham):

 * needs_better_patch:   => 0
 * needs_tests:   => 0
 * needs_docs:   => 0


Comment:

 I can't reproduce on Ubuntu 14.04, Python 3.5.2. Just in case, how many
 parallel process do you have? I have 4 here.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.577fb329c67274d461cb366a57a97a54%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.