Re: Channels: development issue with multiple daphne workers
Thanks Andrew, I tried removing "--fd 0" but it didnt work. I got 504 time out error no matter numprocs=1 or 4. Do you know are there any open-source projects that use channel2? I just want to take a look at their repos as my examples. Thanks Drew On Tuesday, November 27, 2018 at 10:26:16 AM UTC-7, Andrew Godwin wrote: > > I'm not sure why you're getting the _UnixSelectorEventLoop - that sounds > like a Twisted error you should search around for, and make sure you have > the right package versions. > > --fd 0 should not be needed in that command line, since you're already > passing the UNIX socket, I don't remember why it's there. > > I suggest you remove fd=0 and see if that then works with multiple > processes. > > Andrew > > On Mon, Nov 26, 2018 at 3:10 PM Zhiyu (Drew) Li > wrote: > >> Hi there, >> >> I am trying to migrate a tornado project to django channel2. I have moved >> all essential parts and wired them up in channel. It runs OK in development >> mode, but in production it seems the multi-daphne worker configuration is >> causing strange errors. >> AttributeError: '_UnixSelectorEventLoop' object has no attribute >> 'remove_timeout' >> >> I followed instructions on the official doc ( >> https://channels.readthedocs.io/en/latest/deploying.html) to set up >> supervisor and nginx. But I am still lack of understanding how it works. >> Cloud you please explain more on the following settings? >> >> >> *# TCP socket used by Nginx backend upstream* >> *socket=tcp://localhost:8000* >> >> *# Each process needs to have a separate socket file, so we use >> process_num* >> *# Make sure to update "mysite.asgi" to match your project name* >> *command=daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 >> --access-log - --proxy-headers mysite.asgi:application* >> >> *# Number of processes to startup, roughly the number of CPUs you have* >> *numprocs=4* >> >> *Questions:* >> What is "--fd 0" here? Is "0" the file descriptor for socket >> "tcp://localhost:8000" that Nginx is proxying to? >> What is "-u /run/daphne/daphne%(process_num)d.sock" then? Why do we need >> to set up separate socket for each daphne process? Are all the daphne >> processes already talking to "fd 0"? >> >> My website only works if I change either of the following >> A) change to numprocs=1 ; >> Or >> B) remove "-u /run/daphne/daphne%(process_num)d.sock" >> >> Thanks >> Drew >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAMmsbU%3D-oPX_yZ7Kx8PmVfjwKJNSSC0hKOEUXyRGztzHogKGzA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/django-users/CAMmsbU%3D-oPX_yZ7Kx8PmVfjwKJNSSC0hKOEUXyRGztzHogKGzA%40mail.gmail.com?utm_medium=email_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/9657cf08-ced0-49d2-ab83-8338ebe1095a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Channels: development issue with multiple daphne workers
Hi there, I am trying to migrate a tornado project to django channel2. I have moved all essential parts and wired them up in channel. It runs OK in development mode, but in production it seems the multi-daphne worker configuration is causing strange errors. AttributeError: '_UnixSelectorEventLoop' object has no attribute 'remove_timeout' I followed instructions on the official doc ( https://channels.readthedocs.io/en/latest/deploying.html) to set up supervisor and nginx. But I am still lack of understanding how it works. Cloud you please explain more on the following settings? *# TCP socket used by Nginx backend upstream* *socket=tcp://localhost:8000* *# Each process needs to have a separate socket file, so we use process_num* *# Make sure to update "mysite.asgi" to match your project name* *command=daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers mysite.asgi:application* *# Number of processes to startup, roughly the number of CPUs you have* *numprocs=4* *Questions:* What is "--fd 0" here? Is "0" the file descriptor for socket "tcp://localhost:8000" that Nginx is proxying to? What is "-u /run/daphne/daphne%(process_num)d.sock" then? Why do we need to set up separate socket for each daphne process? Are all the daphne processes already talking to "fd 0"? My website only works if I change either of the following A) change to numprocs=1 ; Or B) remove "-u /run/daphne/daphne%(process_num)d.sock" Thanks Drew -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAMmsbU%3D-oPX_yZ7Kx8PmVfjwKJNSSC0hKOEUXyRGztzHogKGzA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Channels: Test case fails on async.sleep(X>0)
Hi there,
I was trying to write a test for a consumer. It is just a very simply
AsyncHttpConsumer subclass that awaits on asyncio.sleep(3) and returns a
"OK" in plain text.
The test case is:
@pytest.mark.asyncio
async def test_my_consumer():
communicator = HttpCommunicator(BasicHttpConsumer, "GET", "asynchttp")
response = await communicator.get_response()
assert response["status"] == 200
assert response["body"] == b"OK"
1) Error 1:
I run it with command "pytest .py"
The first error I got was:
django.core.exceptions.ImproperlyConfigured: Requested settings, but
settings are not configured. You must either define the environment
variable DJANGO_SETTINGS_MODULE or call settings.configure() before
accessing settings.
I google online and found this fix:
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "MODULE_PATH_TO.settings")
2) Error 2:
The new error I am getting is:
self = , exc_type =
def _do_exit(self, exc_type: Type[BaseException]) -> None:
if exc_type is asyncio.CancelledError and self._cancelled:
self._cancel_handler = None
self._task = None
> raise asyncio.TimeoutError
E concurrent.futures._base.TimeoutError
But If I change async.sleep(3) to async.sleep(0) the test passed. Not sure
why
Also how to test url mapping works correctly?
Thanks
Drew
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAMmsbUk-3AemjcTxxidf4GBvKeWG7b72qwfUcs5owReTeOEMiw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Re: Channel: Migrate regular Django project to Channels2
Thanks. I added a async http consumer from the doc. It kept throwing error on "Header name 'Content-Type' expected to be `bytes`, but got ``". I was able to get it running after changing to (b"Content-Type", b "text/plain"), from channels.generic.http import AsyncHttpConsumer class BasicHttpConsumer(AsyncHttpConsumer): async def handle(self, body): await asyncio.sleep(10) await self.send_response(200, b"Your response bytes", headers=[ (b"Content-Type", b"text/plain"), ]) On Wednesday, November 7, 2018 at 8:56:52 PM UTC-7, Andrew Godwin wrote: > > Channels doesn't take over at all unless you configure an async consumer. > To make sure it's working I'd recommend writing a single async consumer, > and running a test against that to ensure it works. > > Andrew > > On Wed, Nov 7, 2018 at 1:11 PM Zhiyu (Drew) Li > wrote: > >> Hi there, >> >> I am trying to migrate an existing Django project to Channels2. We want >> to keep all existing sync codes unchanged and we will continue mode of the >> development in sync mode. But having Channels opens the opportunity to >> develop some features in async mode. >> >> I was able to install channels2 and get it configured. The portal still >> runs and all tests passed. But does it mean the migration is successful? >> How can I know if the tests were actually run in the channels2 env? >> >> Anything particular I need to double check? like a migration checklist >> >> Thanks >> Drew >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users...@googlegroups.com . >> To post to this group, send email to django...@googlegroups.com >> . >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/CAMmsbU%3DzsYB5J7p4_mC1Gj3HqjiLRLo7Hc4UhaHEN_FXRv0ScA%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/django-users/CAMmsbU%3DzsYB5J7p4_mC1Gj3HqjiLRLo7Hc4UhaHEN_FXRv0ScA%40mail.gmail.com?utm_medium=email_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/1f880fe4-2eab-4a41-bcce-f36f3bc2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Channel: Migrate regular Django project to Channels2
Hi there, I am trying to migrate an existing Django project to Channels2. We want to keep all existing sync codes unchanged and we will continue mode of the development in sync mode. But having Channels opens the opportunity to develop some features in async mode. I was able to install channels2 and get it configured. The portal still runs and all tests passed. But does it mean the migration is successful? How can I know if the tests were actually run in the channels2 env? Anything particular I need to double check? like a migration checklist Thanks Drew -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAMmsbU%3DzsYB5J7p4_mC1Gj3HqjiLRLo7Hc4UhaHEN_FXRv0ScA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Channels: event loop and thread
Hi there, I got the following 2 questions There is only one thread (MainThread) and one event loop running on it by default if all existing consumers are asynchronous consumers? Regular sync views (if any) are handled by Threadpoolexecutor and run in a sub Thread. These sync views won't block the event loop in the main thread. Channels takes care of this subthread and I don't need to explicitly manage it. Right? Thanks Drew -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAMmsbUkjbuRvYjoZDyDY%3DXkaoQJHHiG659XKX-tPCbe7HqgivA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: How to subclass AsyncHttpConsumer for a GET request?
Thanks! Another question it seems once I manually put 'http': MyAsyncHttpConsumer pair in the ProtocolRouter, all existing sync http views stopped running as their type is "http" as well. So how to do a mix of async HTTP and sync http? Thanks Drew On Mon, Oct 29, 2018, 23:15 Andrew Godwin wrote: > Yup, that's the right way - subclass the async consumer class and then > write a handle method. You have to do your own post/get distinctions, like > in a Django view, but with the scope rather than the request. > > Andrew > > On Mon, Oct 29, 2018 at 4:37 PM Zhiyu/Drew Li wrote: > >> Not sure if this is the best way. I just found >> inside AsyncHttpConsumer.handle() I can access self.scope['method'] to >> determine if it is a GET or POST or others. >> >> Thanks >> Drew >> >> >> >> >> >> >> On Monday, October 29, 2018 at 3:50:43 PM UTC-6, Zhiyu/Drew Li wrote: >>> >>> Hi there, >>> >>> Newbie to Channels. >>> >>> I am trying to write a Async consumer to handle a http GET request >>> How to write a subclass MyAsynHttpConsumer(AsyncHttpConsumer) for this >>> purpose? Or I am looking at the wrong class? >>> >>> Also if I understand correctly, I should manually add a new pair 'http': >>> MyAsynHttpConsumer to ProtocolTypeRouter() >>> >>> async_http_urlpatterns = [ >>> url(r'^async-http/$', consumers. MyAsynHttpConsumer ), >>> ] >>> >>> ProtocolTypeRouter ( >>> 'http': AuthMiddlewareStack( >>> URLRouter( >>> my_channels_app.routing.async_http_urlpatterns >>> ) >>> ), >>> ) >>> >>> Thanks >>> Drew >>> >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-users+unsubscr...@googlegroups.com. >> To post to this group, send email to django-users@googlegroups.com. >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-users/8c46871d-6e24-47e1-8813-5721014aedca%40googlegroups.com >> <https://groups.google.com/d/msgid/django-users/8c46871d-6e24-47e1-8813-5721014aedca%40googlegroups.com?utm_medium=email_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "Django users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/django-users/zQcvIvqapCo/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at https://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAFwN1uopgpGn7uASW0%2BteonKN8qQFdBXcwA2N8_JM1-CTYiDdg%40mail.gmail.com > <https://groups.google.com/d/msgid/django-users/CAFwN1uopgpGn7uASW0%2BteonKN8qQFdBXcwA2N8_JM1-CTYiDdg%40mail.gmail.com?utm_medium=email_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAMmsbUnQC-MFDHCV2Py1B_BajRRHDQJGjkOowSMBcNOULfTx%3DA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: How to subclass AsyncHttpConsumer for a GET request?
Not sure if this is the best way. I just found inside AsyncHttpConsumer.handle() I can access self.scope['method'] to determine if it is a GET or POST or others. Thanks Drew On Monday, October 29, 2018 at 3:50:43 PM UTC-6, Zhiyu/Drew Li wrote: > > Hi there, > > Newbie to Channels. > > I am trying to write a Async consumer to handle a http GET request > How to write a subclass MyAsynHttpConsumer(AsyncHttpConsumer) for this > purpose? Or I am looking at the wrong class? > > Also if I understand correctly, I should manually add a new pair 'http': > MyAsynHttpConsumer to ProtocolTypeRouter() > > async_http_urlpatterns = [ > url(r'^async-http/$', consumers. MyAsynHttpConsumer ), > ] > > ProtocolTypeRouter ( > 'http': AuthMiddlewareStack( > URLRouter( > my_channels_app.routing.async_http_urlpatterns > ) > ), > ) > > Thanks > Drew > > > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/8c46871d-6e24-47e1-8813-5721014aedca%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
How to subclass AsyncHttpConsumer for a GET request?
Hi there, Newbie to Channels. I am trying to write a Async consumer to handle a http GET request How to write a subclass MyAsynHttpConsumer(AsyncHttpConsumer) for this purpose? Or I am looking at the wrong class? Also if I understand correctly, I should manually add a new pair 'http': MyAsynHttpConsumer to ProtocolTypeRouter() async_http_urlpatterns = [ url(r'^async-http/$', consumers. MyAsynHttpConsumer ), ] ProtocolTypeRouter ( 'http': AuthMiddlewareStack( URLRouter( my_channels_app.routing.async_http_urlpatterns ) ), ) Thanks Drew -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAMmsbUnBV5x8ZEQ4xDC9v2V7EB%3DvpwnFREhZYTWnc5Xzf93J2w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: Channels -- a question about how ProtocolTypeRouter works
Thanks! So that means some objects last across multiple scopes, and they
can create another obj that only lasts for one specific scope?
I am still confused how the different layers of middleware wrap around the
real asgi application -- a obj just lasts for one scope.
I need to go through the code deeper.
Thanks
Drew
On Fri, Oct 26, 2018 at 4:07 PM Andrew Godwin wrote:
> Hi Drew,
>
> The return value is indeed an object, but if you look closely at it you'll
> see it defines a __call__ method - meaning that when it's called as an ASGI
> application, it then returns a further object to act inside the scope.
>
> Andrew
>
> On Fri, Oct 26, 2018 at 2:55 PM Zhiyu/Drew Li wrote:
>
>> Hi,
>>
>> Newbie here. (please bear with me if this a stupid question)
>>
>> I am going through the tutorial and doc and got this question regarding how
>> ProtocolTypeRouter works.
>> ProtocolTypeRouter.__init__ takes a dict "application_mapping" and append a
>> key-value pair to it: "http': class AsgiHandler
>> As doc says ASGI application is created on a per scope basis, I think I
>> vaguely understand how the "http" one works.
>> Once a http scope comes in, an instance/object of class AsgiHandler is
>> initiated, which lasts for the duration of the scope.
>> So it is one scope leads to one ASGI application/object
>>
>> But I am confused about this one:
>> application = ProtocolTypeRouter({
>> # (http->django views is added by default)
>> 'websocket': AuthMiddlewareStack(
>> URLRouter(
>> chat.routing.websocket_urlpatterns
>> )
>> ),
>> })
>> The whole value of this key-value pair
>> AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns)) seems to
>> be a object not a class,
>> so that means the application already exists before a websocket scope comes
>> in. How does this conform to "one scope one application" rule?
>>
>> Any help would be appreciated.
>>
>> Thanks
>> Drew
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-users+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-users@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-users/f6088bd4-457a-4106-b883-332af9a99337%40googlegroups.com
>> <https://groups.google.com/d/msgid/django-users/f6088bd4-457a-4106-b883-332af9a99337%40googlegroups.com?utm_medium=email_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/CAFwN1urusiZjfH2HSnkaRcuNVUXMgLuLRxs%3DxH9tK%3D35enFDow%40mail.gmail.com
> <https://groups.google.com/d/msgid/django-users/CAFwN1urusiZjfH2HSnkaRcuNVUXMgLuLRxs%3DxH9tK%3D35enFDow%40mail.gmail.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAMmsbU%3D0nwz44EzCnMriL2FS-oOL7LyrYOJpp5ocPO3i%3Dj5CFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Channels -- a question about how ProtocolTypeRouter works
Hi,
Newbie here. (please bear with me if this a stupid question)
I am going through the tutorial and doc and got this question regarding how
ProtocolTypeRouter works.
ProtocolTypeRouter.__init__ takes a dict "application_mapping" and append a
key-value pair to it: "http': class AsgiHandler
As doc says ASGI application is created on a per scope basis, I think I vaguely
understand how the "http" one works.
Once a http scope comes in, an instance/object of class AsgiHandler is
initiated, which lasts for the duration of the scope.
So it is one scope leads to one ASGI application/object
But I am confused about this one:
application = ProtocolTypeRouter({
# (http->django views is added by default)
'websocket': AuthMiddlewareStack(
URLRouter(
chat.routing.websocket_urlpatterns
)
),
})
The whole value of this key-value pair
AuthMiddlewareStack(URLRouter(chat.routing.websocket_urlpatterns)) seems to be
a object not a class,
so that means the application already exists before a websocket scope comes in.
How does this conform to "one scope one application" rule?
Any help would be appreciated.
Thanks
Drew
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/f6088bd4-457a-4106-b883-332af9a99337%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
