RQ not picking up tasks

[Mukul Mantosh]

Hi,

I have a serious issue which occurs very rarely. I was running RQ workers 
and it was picking up tasks smoothly. Suddenly i increased the numprocs in 
supervisor configuration from 2 to 3 and i restarted the supervisor. After 
that restart no tasks are getting picked by the RQ worker. This problems 
happens very rarely and i find that RQ is not picking up tasks.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/896f0f3a-2605-45fd-8838-36564334807e%40googlegroups.com.

sending notification to one user using Channels 2

[Mukul Mantosh]

I want to send notification to specific authenticated user using Channels 2.


Stackoverflow: 
https://stackoverflow.com/questions/55310717/sending-notification-to-one-user-using-channels-2



-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/eb7c7c52-3f2e-421e-9de1-26b67a998c18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Deploying a chatbot with Django: WebSockets vs HTTP

[Mukul Mantosh]

I currently have a chatbot app in my django project that I would like to 
deploy.


I am confused as to whether I should use WebSockets or normal HTTP calls 
(AJAX) for it's implementation. This is what I have understood about the 
pros/cons of each(in the context of my use-case) till now after some 
internet research.
Reasons for using WebSockets over HTTP:
   
   1. WebSockets are recommended for chat applications because of low 
   overhead per user message once the connection has been established. HTTP 
   will have the overhead of establishing a connection each time a message is 
   sent (Also the header will be bigger in size, which will be sent with each 
   request/user message)
   2. WebSockets allow for real-time communication without *workarounds* like 
   polling in case of HTTP which can lead to many unnecessary requests from 
   the client.

Reasons for using HTTP over WebSockets:
   
   1. HTTP *just might* be okay for a chatbot(not human to human chat) 
   because the response of the chatbot is only triggered when the user 
   messages something i.e. The Bot isn't expected to message the user all of a 
   sudden at random intervals which would require real-time communication. The 
   Bot only sends replies to user messages.
   2. Will have to use django-channels and an ASGI server for the chatbot 
   part, which will require maintaining two servers (along with WSGI), which 
   will make the implementation a little bit complicated.

What should I go ahead with?

P.S. Someone suggested I look into services like Pusher 
, although it doesn't seem to have server side SDK for 
python

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d7611fcc-76a7-4533-9b2b-fa21891dd42d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Creating Reports in Django Admin Panel

[Mukul Mantosh]

How to create a report in  django-admin panel of number of users added to 
the database in a day, week and month. Also, how many API calls for every 
particular API has been made in a day, week and month.

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/cde57aba-e168-4c74-97b9-7e4b3a48458c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Handling Celery Connection Lost Problem

[Mukul Mantosh]

I tried googling Matemática A3K but did not find any proper response but 
finally i came to know that i need to manage high availability redis or 
rabbitmq clusters. If there is any such option given by celery to simply 
skip the task if any broker fails then it would have solved my problem.

Thanks for helping me out @Matemática , @Jani Tianen , @Jason.

On Saturday, January 6, 2018 at 12:13:19 PM UTC+5:30, Matemática A3K wrote:
>
>
>
> On Fri, Jan 5, 2018 at 9:01 PM, Jason > 
> wrote:
>
>> To reinforce on what Jani Tianen said, this is not a django or python 
>> issue, nor really a Celery issue.  What you should research and investigate 
>> is high availability rabbitmq clusters, if this is such a concern for you.
>>
>> Indeed, Mukul, you should google first :)
>  
>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/bd85fa9c-1917-45dd-84cd-b86a94993098%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/df3e70a7-c3e9-42a6-8fc0-1ae6947c9e95%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Handling Celery Connection Lost Problem

[Mukul Mantosh]

@Matemática A3K  

*If you restart rabbitmq at this point, does it works? *

After restarting rabbitmq everything is working fine. This example simply 
tells that whenever rabbitmq is down your code is going to stuck, so we 
need to design high available rabbitmq server which will be available 
during heavy workloads.

Is there any solution to this problem because it is a common scenario when 
a broker might fail.

On Thursday, January 4, 2018 at 1:04:06 AM UTC+5:30, Matemática A3K wrote:
>
>
>
> On Wed, Jan 3, 2018 at 6:25 AM, Mukul Mantosh  > wrote:
>
>> *I'm not sure about understanding you, where are you trying to call it 
>> again? from ipython?*
>>
>> I am calling from Django, the code is inside the view.
>>
>> *def test(request):*
>> *  try:*
>> *add.delay(2, 2)*
>> *  except add.OperationalError as exc:*
>> *print('error')*
>> *return HttpResponse('working')*
>>
>> Point 1 - Stop RabbitMQ Server from terminal manually. (sudo service 
>> rabbitmq-server stop).
>> Point 2 - Reload the view from the browser, it will immediately throw 
>> connection refused error which i could catch easily using try except block 
>> as provided in above example.
>> Point 3 - If you try again reloading the page it hangs up over there. and 
>> it won't send any http response because it is still waiting and waiting to 
>> get response.
>>
>
> I think this might be because celery can't enqueue the task, so it does 
> not "return" to django 
>  
>
>> Point 4 - You can see celery in background trying to reconnect to the 
>> broker every 5-10 seconds.
>>
>
> If you restart rabbitmq at this point, does it works?
>  
>
>>
>>
>> *For what I understood, your problem is assuming that if a celery task 
>> fails, it won't be retried :)*
>>
>> It should be retried but i can't make a user wait for it because signup 
>> process is very quick and we cannot halt any user because our connection is 
>> lost. I just simply want if connection lost try for 10 seconds if connected 
>> its good otherwise just move on don't get stuck.
>>
>> Even if i tried add.apply_async((2, 2), retry=False) but it is still  
>> not working and the page is not giving back the HTTP Response. 
>>
>> I think i have cleared everything what you wanted to know @Matemática A3K
>> .
>>
>
> https://github.com/celery/celery/issues/3241
> https://github.com/celery/celery/issues/3921
>
> Seems that it is retried by celery, but with a missing rabbitmq broker it 
> won't work at all. If you take away the broker, you are taking away how it 
> works, there is no way of delivering messages. The problem is how to ensure 
> the broker is always working.
>
> Again, this seems more appropriate for the celery community :)
>
>
>>
>> On Wednesday, January 3, 2018 at 2:08:24 PM UTC+5:30, Matemática A3K 
>> wrote:
>>>
>>>
>>>
>>> On Tue, Jan 2, 2018 at 11:14 PM, Mukul Mantosh  
>>> wrote:
>>>
>>>> I am not using result backend my question is that when the broker 
>>>> connection is lost it throws a connection refused exception which i could 
>>>> normally catch through the following given below code.
>>>>
>>>> *try:*
>>>> *  add.delay(2, 2)*
>>>> *except add.OperationalError as exc:*
>>>> *  print('error');*
>>>>
>>>> Reference 1: 
>>>> http://docs.celeryproject.org/en/latest/userguide/calling.html#connection-error-handling
>>>> Reference 2: https://github.com/celery/celery/issues/3933
>>>>
>>>>
>>>> This try except block works only one time and next time when i try to 
>>>> call again add.delay(2,2)the code is waiting to execute because celery 
>>>> is re-trying to establish the connection with the broker.
>>>>
>>>>
>>> I'm not sure about understanding you, where are you trying to call it 
>>> again? from ipython?
>>>  
>>>
>>>> I just simply don't want to do, for example: There is a website where a 
>>>> user signup as a new user and we have to send an verification email 
>>>> through 
>>>> celery and suddenly the connection gets lost then the code will be in 
>>>> waiting state because celery is again retrying to establish a connection 
>>>> with the lost broker.
>>>>
>>>>
>>> I agree that this may be better for a celery mailing list, a

Re: Handling Celery Connection Lost Problem

[Mukul Mantosh]

On Tuesday, January 2, 2018 at 9:16:33 AM UTC+5:30, Mukul Mantosh wrote:
>
> How to handle the problem when celery is unable to connect to the broker 
> (redis or rabbitmq)...while the connection gets lost the entire code gets 
> stuck because celery is trying to reconnect to the host.
>
> But without being stuck how can i simply bypass this problem and my code 
> be still running smoothly.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d051b09f-73ce-4d7f-b699-bb96795f785c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Handling Celery Connection Lost Problem

[Mukul Mantosh]

*I'm not sure about understanding you, where are you trying to call it 
again? from ipython?*

I am calling from Django, the code is inside the view.

*def test(request):*
*  try:*
*add.delay(2, 2)*
*  except add.OperationalError as exc:*
*print('error')*
*return HttpResponse('working')*

Point 1 - Stop RabbitMQ Server from terminal manually. (sudo service 
rabbitmq-server stop).
Point 2 - Reload the view from the browser, it will immediately throw 
connection refused error which i could catch easily using try except block 
as provided in above example.
Point 3 - If you try again reloading the page it hangs up over there. and 
it won't send any http response because it is still waiting and waiting to 
get response.
Point 4 - You can see celery in background trying to reconnect to the 
broker every 5-10 seconds.


*For what I understood, your problem is assuming that if a celery task 
fails, it won't be retried :)*

It should be retried but i can't make a user wait for it because signup 
process is very quick and we cannot halt any user because our connection is 
lost. I just simply want if connection lost try for 10 seconds if connected 
its good otherwise just move on don't get stuck.

Even if i tried add.apply_async((2, 2), retry=False) but it is still  not 
working and the page is not giving back the HTTP Response. 

I think i have cleared everything what you wanted to know @Matemática A3K.


On Wednesday, January 3, 2018 at 2:08:24 PM UTC+5:30, Matemática A3K wrote:
>
>
>
> On Tue, Jan 2, 2018 at 11:14 PM, Mukul Mantosh  > wrote:
>
>> I am not using result backend my question is that when the broker 
>> connection is lost it throws a connection refused exception which i could 
>> normally catch through the following given below code.
>>
>> *try:*
>> *  add.delay(2, 2)*
>> *except add.OperationalError as exc:*
>> *  print('error');*
>>
>> Reference 1: 
>> http://docs.celeryproject.org/en/latest/userguide/calling.html#connection-error-handling
>> Reference 2: https://github.com/celery/celery/issues/3933
>>
>>
>> This try except block works only one time and next time when i try to 
>> call again add.delay(2,2)the code is waiting to execute because celery 
>> is re-trying to establish the connection with the broker.
>>
>>
> I'm not sure about understanding you, where are you trying to call it 
> again? from ipython?
>  
>
>> I just simply don't want to do, for example: There is a website where a 
>> user signup as a new user and we have to send an verification email through 
>> celery and suddenly the connection gets lost then the code will be in 
>> waiting state because celery is again retrying to establish a connection 
>> with the lost broker.
>>
>>
> I agree that this may be better for a celery mailing list, as "low level" 
> celery is probably not the main expertise of most of the people reading 
> here, as Jason said, for what I understand 
> http://docs.celeryproject.org/en/latest/userguide/calling.html#calling-retry 
> should be sufficient for the "normal" cases.
>
> Have in mind that exception will be raised at the celery level, not django 
> because it's async, 
> http://docs.celeryproject.org/en/latest/userguide/calling.html#linking-callbacks-errbacks.
>  
> Django delegates the task to celery and celery takes care of executing it. 
> If you need to perfect warranty of execution, do it sync, inside the django 
> view send the verification mail, where you can show to the user that the 
> confirmation mail hasn't been sent.
>  
>
>> How can we solve this problem ?
>>
>>
> For what I understood, your problem is assuming that if a celery task 
> fails, it won't be retried :)
>  
>
>>
>>
>> On Wednesday, January 3, 2018 at 12:45:10 AM UTC+5:30, Jason wrote:
>>>
>>> With a broker connection loss, the only thing that will happen is your 
>>> workers won't pick up new tasks.
>>>
>>> If you're posting to a result backend like redis and lose the 
>>> connection, then an exception will be raised in the logs and the task will 
>>> shut down.
>>>
>>> Remember tasks are independent processes and you can tell each worker 
>>> how many tasks to execute before its process is killed.
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@goo

Re: Handling Celery Connection Lost Problem

[Mukul Mantosh]

I am not using result backend my question is that when the broker 
connection is lost it throws a connection refused exception which i could 
normally catch through the following given below code.

*try:*
*  add.delay(2, 2)*
*except add.OperationalError as exc:*
*  print('error');*

Reference 1: 
http://docs.celeryproject.org/en/latest/userguide/calling.html#connection-error-handling
Reference 2: https://github.com/celery/celery/issues/3933


This try except block works only one time and next time when i try to call 
again add.delay(2,2)the code is waiting to execute because celery is 
re-trying to establish the connection with the broker.

I just simply don't want to do, for example: There is a website where a 
user signup as a new user and we have to send an verification email through 
celery and suddenly the connection gets lost then the code will be in 
waiting state because celery is again retrying to establish a connection 
with the lost broker.

How can we solve this problem ?



On Wednesday, January 3, 2018 at 12:45:10 AM UTC+5:30, Jason wrote:
>
> With a broker connection loss, the only thing that will happen is your 
> workers won't pick up new tasks.
>
> If you're posting to a result backend like redis and lose the connection, 
> then an exception will be raised in the logs and the task will shut down.
>
> Remember tasks are independent processes and you can tell each worker how 
> many tasks to execute before its process is killed.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/41026c20-3ca0-46bd-8bab-361eba75aadc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Handling Celery Connection Lost Problem

[Mukul Mantosh]

I am not losing connection but i am preparing for failure if what happens 
broker connection is lost and how celery will handle the task...i don't 
want the request to stuck infinite time.

On Tuesday, January 2, 2018 at 6:24:22 PM UTC+5:30, Jason wrote:
>
> This is probably something better suited for the celery github issue 
> tracker or https://groups.google.com/forum/#!forum/celery-users
>
> how come you're losing connections so often?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/0f2b5b0c-4f5e-4bee-8f1f-5ec78bb91053%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Handling Celery Connection Lost Problem

[Mukul Mantosh]

How to handle the problem when celery is unable to connect to the broker 
(redis or rabbitmq)...while the connection gets lost the entire code gets 
stuck because celery is trying to reconnect to the host.

But without being stuck how can i simply bypass this problem and my code be 
still running smoothly.


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/852919f5-8f4e-4f75-a139-d88de495738c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Image rename on Upload to S3

[Mukul Mantosh]

How to rename an image with FileField type. Please tell me how to do it in 
*views* while uploading an image to S3. 

import osfrom uuid import uuid4
def path_and_rename(path):
def wrapper(instance, filename):
ext = filename.split('.')[-1]
# get filename
if instance.pk:
filename = '{}.{}'.format(instance.pk, ext)
else:
# set filename as random string
filename = '{}.{}'.format(uuid4().hex, ext)
# return the whole path to the file
return os.path.join(path, filename)
return wrapper

FileField(upload_to=path_and_rename('upload/here/'), ...)

*The above code works well in models.py file but throws back error during 
migration process. Please help me how can i solve this issue in views.py file 
because once i place the above code in models.py it works very well but if 
there is any change in model and i do a migration then it throws back an error.*


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/1360d55d-24bb-4464-baac-ac012de048f8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: DRF JWT Token with Mobile

[Mukul Mantosh]

Django Rest Framework JWT 
(http://getblimp.github.io/django-rest-framework-jwt/) as specified that 
JWT token can be obtained by passing username and password.

Example:
$ curl -X POST -d "username=admin&password=password123" http://localhost:
8000/api-token-auth/

How to obtain the JWT Token using only mobile number. 
$ curl -X POST -d "mobile=1234567890" http://localhost:8000/api-token-auth/

Normally JWT works with the combination of username and password but i want 
the token using the mobile number only.


On Thursday, December 28, 2017 at 6:22:07 PM UTC+5:30, Krishnasagar 
Subhedarpage wrote:
>
> Can you elaborate question? Please add some background for it.
> ---
> Krishnasagar
>
>
>
> On Thu, Dec 28, 2017 at 1:40 PM, Mukul Mantosh  > wrote:
>
>> How to get JWT Token in DRF using only mobile number as the parameter 
>> instead of email and password..
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-users...@googlegroups.com .
>> To post to this group, send email to django...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-users.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-users/1b3ae8bb-2039-4aa9-8a9f-2c4ddb2ffc55%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-users/1b3ae8bb-2039-4aa9-8a9f-2c4ddb2ffc55%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/b0d5e227-7488-4c86-b757-681c287a1558%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

DRF JWT Token with Mobile

[Mukul Mantosh]

How to get JWT Token in DRF using only mobile number as the parameter 
instead of email and password..

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/1b3ae8bb-2039-4aa9-8a9f-2c4ddb2ffc55%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Insert data into multiple models

[Mukul Mantosh]

I have two models (User and Employee). I want to create a new user and make 
a reference between employee and user. 


*User Table*
1. Name
2. E-Mail
3. Date of Birth
4. Contact Number


*Employee Table*
1. EmployeeID
2. Address 
3. Country
4. Profile Photo
5. *user_id*

*Problem Statement:  *When i create a new user then automatically it should 
create a new record in employee table with *user_id* referencing to the 
user table and without writing two SQL Statements and should also follow 
the database transaction property(ACID).


-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/124695ae-e1f1-4252-8bbb-272cb438243b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.