Re: django project avoid reload page where work algorithm

[Tom Forbes]

This is the mailing list for the development of Django itself, I would post
this to the Django users mailing list instead if you want any answers to
your question.

It sounds like an ideal use case for celery though.

On 8 Oct 2017 01:41, "Xristos Xristoou"  wrote:

I have a website where the user can be put three numbers on my html
template and get some results from my personal mathematical algorithm. the
result save at user personal table on my database and can see in specific
tab in my website.

my problem is where the algorithm to calculate result maybe take time
between 5-10 minutes in this time the browser stay on reload. if user
change tab or close browser or maybe have problem with internet connection
then loose that request and need again to put the numbers and again wait
for results.

I want after the user request from my form in html to keep this request and
work my algorithm without reload the page and where the algorithm finish
then to send the user some email or message or just need the user visit the
tab of results to see new results.

that I want to avoid the problems where the algorithm is in running and
user loose data or request or time.

is easy to do that using suproccess,celery or RabbitMQ ?

any idea ?

here the code

views.py

def math_alg(request):
if request.method == "POST":
test = request.POST.get('no1')
test = request.POST.get('no3')
test = request.POST.get('no3')
#start algorith
calc_math(no1,no1,no3,result)
instance = rmodel.objects.create(user=request.user,rfield=result)
instance.save
return render(request, 'page.html', {'result': result})

html :

{% csrf_token %}
  op math calculate:
  
  
  
  {{result }}

-- 
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/
msgid/django-developers/937d9c18-85e2-4077-a1fe-06ef992ce47e%40googlegroups.
com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJMVcZ5%3Dby0B%3Dh83OAgy7Og6Oo61h1GXE-8e%2BSxBRZWHgw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

django project avoid reload page where work algorithm

[Xristos Xristoou]

I have a website where the user can be put three numbers on my html 
template and get some results from my personal mathematical algorithm. the 
result save at user personal table on my database and can see in specific 
tab in my website.

my problem is where the algorithm to calculate result maybe take time 
between 5-10 minutes in this time the browser stay on reload. if user 
change tab or close browser or maybe have problem with internet connection 
then loose that request and need again to put the numbers and again wait 
for results.

I want after the user request from my form in html to keep this request and 
work my algorithm without reload the page and where the algorithm finish 
then to send the user some email or message or just need the user visit the 
tab of results to see new results.

that I want to avoid the problems where the algorithm is in running and 
user loose data or request or time.

is easy to do that using suproccess,celery or RabbitMQ ?

any idea ?

here the code

views.py

def math_alg(request):
if request.method == "POST":
test = request.POST.get('no1')
test = request.POST.get('no3')
test = request.POST.get('no3')
#start algorith
calc_math(no1,no1,no3,result)
instance = rmodel.objects.create(user=request.user,rfield=result)
instance.save
return render(request, 'page.html', {'result': result})

html :

{% csrf_token %}
  op math calculate:
  
  
  
  {{result }}

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/937d9c18-85e2-4077-a1fe-06ef992ce47e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Fellow Report - October 7, 2017

[Tim Graham]

Triaged

---

https://code.djangoproject.com/ticket/28667 - Documentation for extending 
UserCreationForm doesn't work with UserAdmin (accepted)

https://code.djangoproject.com/ticket/28669 - 
bool(ungettext_lazy('%(value)d blah', '%(value)d blahs', 'value')) returns 
False (duplicate)

https://code.djangoproject.com/ticket/28671 - DateTimeField: make 
auto_now_add just a default value (wontfix)

https://code.djangoproject.com/ticket/28678 - Multipart parser should 
decode those JSON parts that are flagged as such (duplicate)

https://code.djangoproject.com/ticket/28672 - CheckboxSelectMultiple does 
not correctly bind to the form attribute passed in when rendered outside 
its containing form (worksforme)

https://code.djangoproject.com/ticket/28681 - QuerySet.get() implies LIMIT 
2 (duplicate)

https://code.djangoproject.com/ticket/28683 - GIS Length function generates 
incorrect queries when evaluated multiple times (duplicate)

https://code.djangoproject.com/ticket/28682 - Allow QuerySet.update() to 
return the IDs of the matched rows (wontfix)

https://code.djangoproject.com/ticket/28646 - Migration calls "CREATE 
INDEX" when one already exists when 'unique' field attribute is added 
(worksforme)

https://code.djangoproject.com/ticket/28556 - resolve of i18n_patterns does 
not match default language paths with prefix_default_language=False 
(needsinfo)

Authored

--

https://github.com/django/django-formtools/pull/107 - Added testing for 
Django 2.0

https://github.com/django/django-contrib-comments/pull/126 - Added testing 
for Django 2.0

Reviewed/committed

--

https://github.com/django/django/pull/9116 - Fixed #28642 -- Added caching 
to parse_accept_lang_header().

https://github.com/django/django/pull/9178 - Fixed #28596 -- Fixed 
QuerySet.bulk_create() and cascade deletion crash on Oracle when using more 
than 65535 parameters.
https://github.com/django/django/pull/9186 - Fixed #28665 -- Change some 
database exceptions to NotImplementedError per PEP 249.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a9332ec1-9c17-48be-afde-30a06dc4876b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Refactoring the autoreloader

[Aymeric Augustin]

Hello Tom,

I think the PR is on the right track. Now it needs to be taken beyond the 
"proof of concept" stage.

Once you have code, docs, and if possibly some tests — currently there are few 
due to the difficulty of testing auto reloading, especially edge cases — for 
steps 1 and 2, I can review changes again and help you get it merged.

Since this is a major overhaul, we'll have to think carefully about how the 
behavior changes and possible backwards incompatibilities. That said, the 
autoreloader is a dev tool. Changing it isn't going to break anyone's 
production. This makes the big-bang approach viable.

Thanks for working on this!

-- 
Aymeric.



> On 29 Sep 2017, at 21:03, Tom Forbes  wrote:
> 
> Hello,
> I've been thinking on and off about how to improve the autoreloader 
> implementation and I wanted to gather some feedback on potential solutions.
> 
> For some background, Django uses a fairly basic autoreload implementation 
> that simply polls the last modified time for loaded Python files once a 
> second. While this isn't the most efficient, it does work and has worked 
> quite well for a long time. When running manage.py runserver, the 
> autoreloader will launch a child "manage.py" with the same arguments and this 
> child process actually runs Django and serves requests. To reload, the child 
> process exits with exit code 3 and the parent restarts it. The code is some 
> of the oldest in Django, with a fair bit of it not touched in 9-12 years.
> 
> While it works (and I'm a believer in "if it isn't broke don't fix it") there 
> are some architectural and performance issues with the current code:
> 
> - Polling every second is not very efficient
> - Detecting when the child process has exited during startup (i.e problem in 
> settings.py) is problematic and the code is rather nasty
> - i18n files are 'reloaded' when they change in a rather hacky way (resetting 
> private attributes in another module)
> - There is limited support for extending the current implementation, and 
> there are cases during development where the parent autoreloader will 
> terminate.
> 
> I don't want this email to be too long, so I'm going to summarize what I 
> think would be a good approach to tackling these problems.
> 
> 1. Refactor the current implementation by removing `pyinotify`, redundant 
> python 2 checks and implement a 'file_changed' signal so other parts of 
> Django can react to file changes (i.e the i18n module can handle resetting 
> it's own state).
> 2. Add support for the "watchdog" library as a replacement for pyinotify. 
> Watchdog implements file system notifications for all major platforms using a 
> fairly simple API, so we can remove polling and have instant reloading. Also 
> support Watchman, a notification Daemon from Facebook.
> 3. Add support for more advanced features, like proper handing of startup 
> errors and socket sharing.
> 
> I've got a merge request that implements all three stages as a proof of 
> concept, but I think it's far too much a change to be done at once and should 
> be done carefully stage by stage. One and two are fairly simple to implement, 
> but three requires see careful consideration as to the best approach (this 
> message is long enough already, I don't want to describe them here).
> 
> Does anyone have any feedback on these ideas? Is it worth persuing even if 
> the current implementation works ok-ish?
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAFNZOJMT9qDk-4pKKXSJysEQCmd6CGxMZBYZs_7BQs_WbAqL6g%40mail.gmail.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 

Re: Refactoring the autoreloader

[Tom Forbes]

Thanks for the feedback! Watchdog does include a stat watcher, and I think
we can delegate to that. Currently we turn on the file system notifications
if the library is available, but we should add a '--stat' flag to disable
this it you are running on filesystems that don't support this. At which
point we can delegate to the watchdog stat watcher if available, else
fallback to our current implementation.

There hasn't been much feedback on this, so I'm guessing there isn't much
appetite for these changes. I think improving the current code is a good
idea though, even without the filesystem notifications there is some work
that could be done.

On 3 Oct 2017 00:57, "François Freitag"  wrote:

Hi Tom,

Thank you for the great summary.


> 2. Add support for the "watchdog" library as a replacement for
> pyinotify. Watchdog implements file system notifications for all major
> platforms using a fairly simple API, so we can remove polling and have
> instant reloading. Also support Watchman, a notification Daemon from
> Facebook.

Filesystem polling is required for some setup, such as mounting code using
NFS or rsync, for example using vagrant synced folders[1]. Although it does
not prevent from using the watchdog library, which provides a
PollingObserver[2], I think it's worth keeping that use case in mind. The
PR has a StatReloader which seem able to handle filesystem polling, I would
suggest either keeping it or delegating the polling to watchdog.

[1] https://www.vagrantup.com/docs/synced-folders/nfs.html
[2] https://pythonhosted.org/watchdog/api.html#module-watchdog.
observers.polling

Cheers,
François


On 09/29/2017 12:03 PM, Tom Forbes wrote:

> Hello,
> I've been thinking on and off about how to improve the autoreloader
> implementation and I wanted to gather some feedback on potential solutions.
>
> For some background, Django uses a fairly basic autoreload implementation
> that simply polls the last modified time for loaded Python files once a
> second. While this isn't the most efficient, it does work and has worked
> quite well for a long time. When running manage.py runserver, the
> autoreloader will launch a child "manage.py" with the same arguments and
> this child process actually runs Django and serves requests. To reload, the
> child process exits with exit code 3 and the parent restarts it. The code
> is some of the oldest in Django, with a fair bit of it not touched in 9-12
> years.
>
> While it works (and I'm a believer in "if it isn't broke don't fix it")
> there are some architectural and performance issues with the current code:
>
> - Polling every second is not very efficient
> - Detecting when the child process has exited during startup (i.e problem
> in settings.py) is problematic and the code is rather nasty
> - i18n files are 'reloaded' when they change in a rather hacky way
> (resetting private attributes in another module)
> - There is limited support for extending the current implementation, and
> there are cases during development where the parent autoreloader will
> terminate.
>
> I don't want this email to be too long, so I'm going to summarize what I
> think would be a good approach to tackling these problems.
>
> 1. Refactor the current implementation by removing `pyinotify`, redundant
> python 2 checks and implement a 'file_changed' signal so other parts of
> Django can react to file changes (i.e the i18n module can handle resetting
> it's own state).
> 2. Add support for the "watchdog" library as a replacement for pyinotify.
> Watchdog implements file system notifications for all major platforms using
> a fairly simple API, so we can remove polling and have instant reloading.
> Also support Watchman, a notification Daemon from Facebook.
> 3. Add support for more advanced features, like proper handing of startup
> errors and socket sharing.
>
> I've got a merge request that implements all three stages as a proof of
> concept, but I think it's far too much a change to be done at once and
> should be done carefully stage by stage. One and two are fairly simple to
> implement, but three requires see careful consideration as to the best
> approach (this message is long enough already, I don't want to describe
> them here).
>
> Does anyone have any feedback on these ideas? Is it worth persuing even if
> the current implementation works ok-ish?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com  django-developers+unsubscr...@googlegroups.com>.
> To post to this group, send email to django-developers@googlegroups.com
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/ms
> 

Re: ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

Thank you for the replay Florian, I will think about what you said that 
Django shouldn't make any assumptions about fields outside the form, but my 
intuition 
tells me that there are more staff which we could be retrieved from the 
ModelForm's.

Some of my initial thoughts:

If the ModelForm doesn't care about anything outside the defined fields, 
then this is just a regular Form, the only difference is that it can use 
Models as a short-cut to be defined. (ModelFormFactory?)
If we agree on this (I'm perfectly fine to agree that this could be the 
scope of a ModelForm), then I would probably not expect to see a method 
like `form.save()`, just force me to use `form.cleaned_data` to build my 
instance, make it more clear that the ModelForm is more like a FormFactory 
class instead of throwing me options like `commit=False`. I believe this 
could save a lot of headache to people. If there is no guarantee that an 
instance cannot be saved after the form has been validated, then do not 
give me an option to shoot myself into the foot.

However, why I could expect the ModelForm to be something more...

This is a ModelForm, yes it may sit into the `forms` namespace, but still 
its half `Model` and a half `Form`, pretending to be a form-only class is 
pity for something that do so much heavy-lifting to remove all the tedious 
tasks from us (love it for this). Use all the metadata we have into the 
model to validate the instance. My only concern here actually is not that 
we didn't know what will happen (I think we know that), but how to show the 
user that the developer got messed up, i.e. how to render an error for a 
field that is not rendered to the user? Should this error be propagated to 
a NON-FIELD error or something else? I don't know yet, maybe no, maybe this 
field should not be validated and lead to a server error. But, however, in 
the scope of the problem shown here (a multi-field-level unique constrain) 
this is not an issue, because this is not a field-level error, this is a 
multi-field-unique constrains, so maybe the only thing that needs to be 
changes, is to stop respecting the excluded fields when validating 
multi-field level unique constrains? I feel its perfectly fine to ask for a 
complete model instance at this stage of the validation process (lots of 
methods got passed from the beginning - form.clean() and instance.clean()).

I would love to see more opinions on the topic. Thank you.

On Saturday, October 7, 2017 at 1:55:24 PM UTC+3, Florian Apolloner wrote:
>
> I think the current behaviour is correct. Django shouldn't make any 
> assumptions about fields not in the form since it doesn't know what will 
> happen… In that sense I disagree with the statement:
>
> > From my point of view when I define a ModelForm I imagine that I have to 
> only explain what the user needs to see when the form gets rendered.
>
> To get what you want use something along the lines of: 
> https://gist.github.com/apollo13/ef72035c5f864174ad3c968dec60d88a
>
> Cheers,
> Florian
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/99469dcc-5850-4168-b490-2ca7accf8ebc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ticket #28609 - Making REQUEST_URI available in runserver

[Shai Berger]

Hi Jay,

First, I'd like to say that your post was very nicely done. You did not miss 
any conventions as far as I could see. I believe the only reason you got no 
responses so far is that the issue is relatively fringe, and your message was 
relatively long.

For the subject matter, though, it seems like your use case could be easily 
served by using the stdlib's urllib.parse.quote_plus (or urllib.quote_plus, if 
you're on Python 2) in your WSGI middleware. This should give you the 
parameters of interest in the format you're looking for, without changing 
Django's code, and while still enjoying the other benefits of URL 
normalization. It seems you have considered only alternatives for passing the 
original data into your code, and passed over the idea of reconstructing from 
available data. Or, if this is what you meant by

> [...] custom query string
> processing for this use case, which would be less compatible and may break
> other expected behaviours.

I think you'll need to give a little more details for the argument to be 
convincing.

Of your other comments, this caught my attention:

> *Could be used inappropriately and may need to be cautioned against*
> 
> I don't see why there is any need for it to be documented, it is providing
> optional access to an internal, as is the case with WSGI parameters in
> other web servers. 

If a feature is not documented, it means you should not rely on it. 
Undocumented APIs are usually added because they are required for implementing 
some documented feature; adding undocumented APIs on their own makes little 
sense.

> That aside, I disagree that requiring caution
> invalidates the value in making a tool available to users.

Requiring caution, in and of itself, does not invalidate the value of making a 
tool available to users -- but it does detract from it. We then ask ourselves 
if we can recommend safer alternatives, and I believe I have.

Hope this helps,
Shai.

Re: ModelForm unique validation is not done right IMHO.

[Florian Apolloner]

I think the current behaviour is correct. Django shouldn't make any 
assumptions about fields not in the form since it doesn't know what will 
happen… In that sense I disagree with the statement:

> From my point of view when I define a ModelForm I imagine that I have to 
only explain what the user needs to see when the form gets rendered.

To get what you want use something along the lines of: 
https://gist.github.com/apollo13/ef72035c5f864174ad3c968dec60d88a

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/68d1762e-0541-4ab6-8586-487f440efaaa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

PostgreSQL Partial Indexes package

[Mattias Linnap]

Hi django-developers,

I have written a package that implements PostgreSQL and SQLite partial 
indexes on top of the new class-based 
indexes: https://github.com/mattiaslinnap/django-partial-index
The most common use case is partial unique constraints, but I have a few 
projects where non-unique partial indexes have turned out useful as well.

I have a few questions on how to continue with this:

1. Right now the "where condition" expression is provided as a string, and 
has to be different for PostgreSQL and SQLite in some common cases (for 
example boolean literals). Is there a good abstraction for SQL expressions 
somewhere in Django internals that I could use instead, something similar 
to Q-expressions perhaps? In particular, to add validate_unique() support 
to ModelForms, I would need to be able to extract all fields that are 
mentioned in the where condition.
2. I've seen mentions of a "check constraints" support being in development 
(https://github.com/django/django/pull/7615). Will that include partial 
unique constraints, or is just for the per-column checks?
3. If separate, then it would be nice to one day get partial indexes merged 
into the contrib.postgres package. Do you have any suggestions on what 
needs to happen before that - more test coverage, more contributors, more 
users, or similar?

Best,

Mattias

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2f9b5333-8e6d-4c0f-9bd3-ae1cf79397c2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

ModelForm unique validation is not done right IMHO.

[Todor Velichkov]

I will best explain what I mean with an example. 

Lets say we have a simple model called Book.

class Book(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=255)

class Meta:
unique_together = ('user', 'name')

so `Books` are linked to users, and we have added an unique constrain 
because we don't want a user to have the same book twice. 

Now lets say we want to allow users to create/update books, when we do this 
we would only need their book.name, we already knew who they are in the 
system.
So a simple ModelForm would look like this:

class BookForm(forms.ModelForm):
class Meta:
 model = Book
 fields = ('name',)

Now since i've omitted the `user` field and I can't have a book w/o a user, 
its my responsibility to populate it. Thus when I create new form i will 
always a pass an instance with a user.
But before we do this, lets manually create 1 book for this user.

user = User.objects.get(pk=1)
book1 = Book.objects.create(user=user, name='Harry Potter')

#now lets build our form
book2 = Book(user=user)
form = BookForm(data={'name': 'Harry Potter'}, instance=book2)

if form.is_valid(): #yes the form will be valid
book2 = form.save() #IntegrityError will be raised with UNIQUE 
constraint failed

Now this happens because `ModelForm.validate_unique 
` 
collects a set of fields to pass as an exclude fields into 
`instance.validate_unique`. 

Please, can someone explain me why is this done in such a manner, and what 
will be the correct way approach this problem.

>From my point of view when I define a ModelForm I imagine that I have to 
only explain what the user needs to see when the form gets rendered.
But when the instance is being validated, I would always want to validate 
this instance as a whole (with the fields that I've omitted).



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e5e49072-9d33-4faf-99f7-f7339437c73a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.