Re: Best way to run a shell script, using input value from Django?

[Alex Heyden]

So, a lot of this doesn't have much to do with Django, but I'm not sure how
familiar you are with web server best practices in general, so apologies if
some of this is remedial for you.

If something takes two minutes to finish, you don't want it being handled
on the request handler itself. Split the job off into a separate worker,
then return *something* to the user. Ideally, that something will allow the
user to continue polling for job updates.

Sounds simple, but the details can be obnoxious. First, you have to figure
out how to track the job. Personally, I prefer to not have to watch a
worker directly. I'd rather watch the results come through. For example, if
your script takes two minutes because it's updating a few thousand items
somewhere, I'd rather watch the items change by OS mtime or auto_now on a
model or whatever makes sense in your context. That means less complexity
in my workers and less complexity in my database. You might not be so
lucky. If one user can kick off a dozen or so of these jobs, and they only
do one thing each, you're probably going to have to track Celery workers
individually by sending to the user a job identifier and storing them in
your database with a foreign key out to your User model. Regardless of how
you decide to do it, the job of the initial request handler is to take care
of this piece of the puzzle: how am I going to track this, kick off the
job, send the tracking information to the user.

The script itself is probably pretty easy. A Celery worker using subprocess
from the standard library will be sufficient, with maybe a little bit
watching stdout to figure out how far along the job is if you need to. Have
the worker write its progress to the database row associated with the job,
or use celery-progress if you'd rather use someone else's implementation of
that idea.

In your UI, you'll want some sort of AJAX loop so the user can watch the
job go. This is usually pretty simple too: just an endpoint that queries
the job's status as recorded in the database and returns it without fanfare.

You probably don't want to automatically delete the database entry for the
job once its done, because you don't know if the user was there to see the
end of it. Save that for a periodic task or a response to the user
acknowledging the end of the job run.

On Wed, Jan 2, 2019 at 4:01 PM Chris Robinson  wrote:

>
> Hello,
>
> I'm going to attempt to generalize my question to make this easier to
> answer.
>
> Lets say I have a simple terminal based shell script (e.g.
> multiply_by_two) that takes a number argument, then multiplies that number
> by 2, and returns the result into a result.txt file.
>
> I would enter something like this:
> multiply_by_two -n 6
>
> The -n flag asks what number I would like to multiply by 2. The result of
> this would be a result.txt, containing the number 12 inside.
>
>
> What I would like to do is develop a simple Djano application that
> contains a text field allowing me to input the number 6, then click
> "Submit."
>
> This job will start on the server by running my custom multiply_by_two
> application with my input parameter (6), but when the job is finished and
> the result.txt is available, the browser will automatically download the
> file.
>
> To make this a tad bit more complex, lets say that the job takes 2 minutes
> to run. What would be the best way to monitor the job? Maybe I accidentally
> close the window.
>
> Not looking for anyone to solve this, I'm just new to Django and want to
> know if someone can give me any pointers on where to start. Are there any
> apps existing that will help me not need to write everything from scratch,
> especially if 'monitoring' is needed? Would Celery be ideal for this?
>
> Thanks for any input!
>
> Regards,
> Chris
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To post to this group, send email to django-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/7da4b068-ac58-4c26-a0bc-ed4925f8a4f9%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 

Best way to run a shell script, using input value from Django?

[Chris Robinson]

Hello, 

I'm going to attempt to generalize my question to make this easier to 
answer. 

Lets say I have a simple terminal based shell script (e.g. multiply_by_two) 
that takes a number argument, then multiplies that number by 2, and returns 
the result into a result.txt file. 

I would enter something like this:
multiply_by_two -n 6

The -n flag asks what number I would like to multiply by 2. The result of 
this would be a result.txt, containing the number 12 inside. 


What I would like to do is develop a simple Djano application that contains 
a text field allowing me to input the number 6, then click "Submit."

This job will start on the server by running my custom multiply_by_two 
application with my input parameter (6), but when the job is finished and 
the result.txt is available, the browser will automatically download the 
file. 

To make this a tad bit more complex, lets say that the job takes 2 minutes 
to run. What would be the best way to monitor the job? Maybe I accidentally 
close the window.

Not looking for anyone to solve this, I'm just new to Django and want to 
know if someone can give me any pointers on where to start. Are there any 
apps existing that will help me not need to write everything from scratch, 
especially if 'monitoring' is needed? Would Celery be ideal for this?

Thanks for any input!

Regards,
Chris

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/7da4b068-ac58-4c26-a0bc-ed4925f8a4f9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.