Re: Is there a function that returns common elements in multiple lists
>>> a = {1, 2, 3}
>>> b = {2, 3, 4}
>>> a & b
{2, 3}
On Wednesday, 26 January 2022 at 23:52:05 UTC+11 xaadx...@gmail.com wrote:
> a = [1,2,3,4]
> b = [3,4,5,6]
>
> Convert list a to set
> a_set = set(a)
> print(a_set.intersection(b))
>
> On Wed, Jan 26, 2022, 5:47 PM Muhammad Shehzad wrote:
>
>> Use intersection
>>
>> On Wed, Jan 26, 2022, 4:06 PM bnmng wrote:
>>
>>> Thank you. I think I'll go with sets as advised, although this method
>>> also looks very neat:
>>> intersection = [item for item in list1 if item in list2] found at
>>> https://datagy.io/python-intersection-between-lists/
>>>
>>>
>>>
>>>
>>> On Tuesday, January 25, 2022 at 3:11:10 PM UTC-5 joezep...@gmail.com
>>> wrote:
>>>
On Tue, Jan 25, 2022 at 3:07 PM bnmng wrote:
> Thank you. That set.intersection with list conversion seems like the
> way to go. On your second point, well taken. I'll keep that in mind for
> future Python questions
>
> On Tuesday, January 25, 2022 at 7:46:58 AM UTC-5 bnmng wrote:
>
>> Hello,
>>
>> Is there a built in function that compares two or more lists and
>> returns elements common to all?
>>
>> like this:
>>
>> def in_both( list_a, list_b ):
>> list_c=[]
>> for value in list_a:
>> if value in list_b:
>> list_c.append(value)
>> return list_c
>>
>> Or this:
>> def in_all(list_of_lists):
>> list_a = list_of_lists.pop(0)
>> list_b = []
>> for value in list_a:
>> in_all_lists = True
>> for each_list in list_of_lists:
>> if not value in each_list:
>> in_all_lists = False
>> if in_all_lists:
>> list_b.append(value)
>> return list_b
>>
>> Thank you
>>
> --
>
You received this message because you are subscribed to the Google
> Groups "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-users...@googlegroups.com.
>
To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/4465553d-8aa1-4ab2-9e9c-19c4117d623fn%40googlegroups.com
>
>
> .
>
--
Oussama Chafiqui
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-users...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/fc7b49a3-2ca2-48e1-8556-f742e16fa4c9n%40googlegroups.com
>>>
>>>
>>> .
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/58190fe5-8839-4698-8db2-a20c0a928ab3n%40googlegroups.com.
Re: Eliminating inter-request race conditions
I thought I'd post a little update, as I'm fairly happy with my progress: Here's the repo's readme. I haven't actually pushed the package to pypi so don't try to follow the instructions yet, but any feedback on the README's content is very welcome: https://github.com/nicois/nango/blob/develop/README.md . Hopefully in the next couple of days I'll push the package to pypi so anyone who's interested can make sure it works as advertised. But still I wonder if client-side validation and feedback should be > optional? That is, if I had to do this myself, I'd hope to find a solution > first that works 100 % without client-side effects. This would also help a > lot with correctness and testing. Then I'd put all eye candy on top, but > keeping it strictly optional. The client-side validation is latent in this release. That is, there is some code which will provide client-side validation with websockets, but it is disabled unless explicitly enabled in settings.py. I've put in the beginnings of some automated tests, which I can expand as I proceed. I agree that the server-side data integrity is more important, lower risk and easier to test. > > There are only two ways I can see of to achieve this: use a full-blown > "rowversion" CAS pattern, where there is a dedicated monotonic column on > each table which automatically increases with each update, or the method I > propose/use, where the original form values are provided via the user agent > when the form is POSTed. > > Then this would have to be temper-proof, wouldn't it? > (e.g. using https://itsdangerous.palletsprojects.com ) No, the (current) intention is not to make this tamper-proof. If you think about it, there is no value from trying to protect against an authenticated and authorised user who wants to submit incorrect values. While there could be HMACs etc, I don't see any value at all, as a malicious user does not need to tamper with the original values to submit bad data. > It might even be possible to serialize the entire state of the object into > a single hidden field and sign it on GET, then check the signature and > deserialize on POST. Or maybe, depending on the exact requirements, even > the checksum of the old state would be enough in order to detect that > something changed between the old version of the model (as it was when the > user started editing it) and the current version (at the time the POST > request arrives). This would roughly correspond to a version number without > requiring an explicit field on the model. > Remember also that there is little to no value in checking that fields on the model have changed, if those fields are not shown on the model: firstly because the form will not update those fields in the database, and secondly because if only those "invisible" fields change, the end-user would see the same thing on the form, and would not alter their behaviour. The value here is in showing a user that one of the fields they are in the process of editing has changed while they have had the form open, and ensuring that they do not accidentally clobber someone else's changes. Thanks for your feedback so far. Nick -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/a449bb5f-c834-457e-83f5-02eec2e0e784n%40googlegroups.com.
Re: Eliminating inter-request race conditions
Good to hear from you Carsten. Thanks in advance for your comments. I'll see what I can address now: > > > *2)* Not being able to safely lock a model/queryset beyond the lifetime > of the request. > I don't quite understand this. It sounds like pessimistic locking? Correct. To be clear, I am not advocating this behaviour by default, but making it as seamless as possible to enable when required, rather than needing to attempt to hand-roll safe locking semantics each time it's needed. > > *3)* Not knowing that data has changed on the server until you submit a > form. > > *4)* Smarter form validation > > Well, this comes at the cost of the complexity to implement this. For me, > at least at this time, the cost is much to high to consider this. > Certainly there is increased complexity. For the websites I am involved in (primarily health-related ones), if I don't end up providing a django-based solution, product owners end up demanding a SPA-based solution or similar, with the even-greater complexity to the development stack, not to mention testing. > > *The solutions* > > > > *Enhanced forms* > > - when rendering a form (using e.g. as_p()), alongside the normal INPUT > DOM elements, include additional hidden fields which store a copy of each > form field's initial value. > > I don't think that having hidden fields with the initial values is > necessary: In your view, you can initialize the form like this: > > if request.method == 'POST': > form = SomeForm(request.POST, initial=init) > ... > > Note the `initial` parameter: It is used just as it is in the GET request. > This allows you to use `form.changed_data` and `form.has_changed()` in form > validation. But where does "init" come from? How can you know what version of the model instance was shown to the user when the form was rendered? There are only two ways I can see of to achieve this: use a full-blown "rowversion" CAS pattern, where there is a dedicated monotonic column on each table which automatically increases with each update, or the method I propose/use, where the original form values are provided via the user agent when the form is POSTed. I guess a third option would be to cache the form values server-side using redis each time a form a served, and provide an ID to it, perhaps even using the CSRF token as the key. Perhaps I am missing something - if there is a way to retrieve the initial value of the form automatically, I would love to use it. Regarding formsets, the same applies, I agree it needs to support that, and by embedding the original values into each form in the formset, it should correctly respect those values when the formset is submitted. If it's of any assistance, I can push some code and provide some examples for you to try out. The same app I am using now should be quite easy to clone and evaluate. > > Best regards, > Carsten Gute Rutsch. Nick -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/cafc7989-44de-4035-88d1-c9d010946ed3n%40googlegroups.com.
"include super" with django templates?
I would like to extend a django template which is used via the include
macro. That is, I want to add additional output before or after what would
otherwise be emitted.
With the normal {% block %} syntax, it's easy to use super to follow the
template heirachy similar to __mro__ - but to my knowledge there isn't an
equivalent for include().
For example, form.as_p() calls render() with a template (by default,
django/forms/p.html). I want my application to modify this, so whenever
p.html is rendered, it is wrapped by my application's supplied template.
(As I want this to work for any form instance, I cannot just override the
default template name)
Another example is
say django/forms/templates/django/forms/widgets/number.html. It consists of
a single include() statement pointint to input.html. I would like to
override input.html, but in a way which eventually renders the original
template, in addition to my application's template.
Any suggestions?
Thanks.
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/9ab15060-2dcd-44a0-abe3-eaeed8015813n%40googlegroups.com.
Re: Automatic subdomain for each user
Once you have the TLS certificates and routing working, can't you make a
tiny piece of middleware which takes the hostname (provided via
nginx/apache in a request header) and add an attribute on the request which
corresponds to your virtual site?
e.g. (dodgy untested code follows):
def multisite_middleware(get_response): def middleware(request):
domain_name = request.META.get('HTTP_HOST', DEFAULT_DOMAIN)
request.my_site = Site.objects.get(id=domain_name) response =
get_response(request) return response return middleware
Then anywhere in your code you can access request.my_site for the
site-specific data you need.
In addition, if you haven't already, take a look
at https://docs.djangoproject.com/en/4.0/ref/contrib/sites/
On Friday, 31 December 2021 at 07:03:44 UTC+11 adigun...@gmail.com wrote:
> Thank you for your detailed response. Following your response point by
> point with few additional googling, it works as expected.
>
> On cpanel:
> I created a subdomain called *
> Then I chose document root as the main Django root. (This is needed so
> that letsencrypt can automatically create a certificate.)
>
> Then the routing works seemlessly
>
> Thank you
>
> On Thu, Dec 30, 2021, 7:06 PM Sherif Adigun wrote:
>
>> Thank you very much for you guidance. I have been able to make it work
>> using Django-hosts on my local. Still trying to fix it out to work on my
>> shared hosting if possible
>>
>> On Thu, Dec 30, 2021, 10:29 AM Sherif Adigun wrote:
>>
>>> Thank you @Tim. Django subdomains looks promising but it's outdated. It
>>> uses codes and imports of django< v2 I have checked django-hosts but it
>>> looks as if it can be used for only predefined list of subdomains. Please
>>> help clarify this doubt.
>>>
>>> Thank you for your time
>>>
>>> On Thursday, December 30, 2021 at 1:10:17 AM UTC+1 Tim Chase wrote:
>>>
On 2021-12-29 15:02, Sherif Adigun wrote:
> I am faced with a requirement where each user is required to use
> the application under his own subdomain. Whenever a user registers,
> he gets username.domain.com and he can add staff, manage reports,
> etc under his subdomain.
>
> What is the best approach to you can suggest please?
Generally this requires
1) configuring your DNS to allow for wildcard domains so that
*.domain.com points at your web server, usually looking something like
*.example.com. 3600 IN A 198.51.100.17
2) setting up your HTTPS certificates. This could be
- involve an ACME-aware Django route (I'm not sure if such a beast
exists)
- configuring an ACME client like certbot to update your
DNS records so that Let's Encrypt can verify that you own the
domain and then grant your a wildcard cert
- use a different CA to obtain a wildcard cert
2b) if you used a more manual method, put the certs in the
appropriate place for your web-server to pick them up
3) configure your web-server front-end/proxy (usually nginx or
Apache, but there are others) to handle the wildcard domains and pass
them through to your Django app or WSGI server or what have you. Make
sure the domain information is passed through in the HTTP_HOST header
4) use django-subdomains[1] (or maybe django-hosts[2]) middleware to
extract the intended host-name from the HTTP_HOST header and possibly
make the route reversable so that .reverse() works
5) check your views so that they make use of the HTTP_HOST
information to filter for user-specific information
-tkc
[1] https://django-subdomains.readthedocs.io/en/latest/
[2] https://github.com/jazzband/django-hosts
--
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "Django users" group.
>>> To unsubscribe from this topic, visit
>>> https://groups.google.com/d/topic/django-users/CTynQlthabY/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> django-users...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-users/6057efaf-5e10-498a-ad2a-40132425412dn%40googlegroups.com
>>>
>>>
>>> .
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/7cef164d-0f9c-4ad9-9d2d-18bfcb2d404bn%40googlegroups.com.
Re: Eliminating inter-request race conditions
Thanks for the reference Carsten. I believe the approach I am taking, regarding optimistic locking, is superior to what is proposed in that thread. Specifically: - there is no need for a special version field to be added to the model - because forms only update specific fields in the associated model(s), there is no point in unnecessarily invalidating the form submission if unrelated model fields have been changed in the background; if a form does not show a field to begin with, having its value change would not alter the behaviour a person would make when editing the form. Really I think this should be a code part of Django: just as you include CSRF protection virtually everywhere, you would be doing this. It is very rare that this feature would be undesired, and many Django developers will be getting burnt by the lack of CAS-like behaviour. I'll continue to tinker with this either way, if only to use internally, if there isn't general interest. cheers, Nick. On Tuesday, 28 December 2021 at 21:35:09 UTC+11 carste...@cafu.de wrote: > Hi Nick, > > maybe this is a case for optimistic locking? > Does the thread at > https://groups.google.com/d/msg/django-users/R7wJBTlC8ZM/MIzvYkWyCwAJ > help? > > Best regards, > Carsten > > > Am 27.12.21 um 06:36 schrieb Nick Farrell: > > Hi all. > > > > I've been using Django for quite a number of years now, in various ways. > Most of the time, I find myself needing to create custom solutions to solve > what appears to be a very common problem. > > > > During the Christmas downtime, I decided to scratch this itch, and am > putting together what will hopefully turn into a solution to what I'll > describe below. I'm writing this here to get a sense of what the Django > community sees in this: is this a niche problem, is it shared by a few > others, or is the lack of these features a fundamental overnight in the > core Django product? > > > > *The problems *(from highest to lowest priority)*:* > > * > > * > > *1)* a form is rendered, the data is changed by a different > task/request, then the form is submitted, overwriting the recent changes. > > > > Whenever models can be modified by multiple users (or even the same user > in different windows/tabs of their browser), this can happen. Also, if > there are any background processes which can modify the data (e.g. celery, > or various data synchronisation services), it's possible. > > In some situations this is no big deal, as the users do not really care, > or you know that the latest data would overwrite the previous data anyway. > But in general, this is a major risk, particularly when dealing with any > health or financial data. > > > > *2)* Not being able to safely lock a model/queryset beyond the lifetime > of the request. > > > > This is related to problem 1, and solving problem 2 may in some > circumstances solve problem 1 - but not always. For example, depending on > how the lock is implemented, a "rogue" task/request may bypass the locking > mechanism and force a change to the underlying data. Also, if a lock is > based on a session, a user may have multiple tabs open in the same browser, > using the same session state (via shared cookies) > > > > Solving this problem will reduce the chance that when a person does post > a form update, that there is any conflict, meaning fewer tears. > > > > *3)* Not knowing that data has changed on the server until you submit a > form. > > > > Ideally there would be a means for someone viewing/editing a form to > immediately be notified if data changes on the server, obsoleting the > current form. This reduces the amount of wasted time is spent completing a > form which is already known to be out of sync, and will need to be redone > anyway (as long as problem 1 is solved; otherwise, there'll be data loss) > > > > *4)* Smarter form validation > > > > There are three types of missing validation: > > - the first is that the default widgets do not support even very simple > client-side validation. For example, a text field might need to match a > regular expression. > > - the second type is an ability to provide (in the model definition) > arbitrary javascript which can be executed client-side to provide richer > realtime validation during data entry. > > - the third type involves effectively providing provisional form data to > the server, and having Django validate() the form content without actually > saving the result. This would allow (for example) inter-field dependencies > to be evaluated without any custom code, providing near-realtime feedback > to
Eliminating inter-request race conditions
Hi all. I've been using Django for quite a number of years now, in various ways. Most of the time, I find myself needing to create custom solutions to solve what appears to be a very common problem. During the Christmas downtime, I decided to scratch this itch, and am putting together what will hopefully turn into a solution to what I'll describe below. I'm writing this here to get a sense of what the Django community sees in this: is this a niche problem, is it shared by a few others, or is the lack of these features a fundamental overnight in the core Django product? *The problems *(from highest to lowest priority)*:* *1)* a form is rendered, the data is changed by a different task/request, then the form is submitted, overwriting the recent changes. Whenever models can be modified by multiple users (or even the same user in different windows/tabs of their browser), this can happen. Also, if there are any background processes which can modify the data (e.g. celery, or various data synchronisation services), it's possible. In some situations this is no big deal, as the users do not really care, or you know that the latest data would overwrite the previous data anyway. But in general, this is a major risk, particularly when dealing with any health or financial data. *2)* Not being able to safely lock a model/queryset beyond the lifetime of the request. This is related to problem 1, and solving problem 2 may in some circumstances solve problem 1 - but not always. For example, depending on how the lock is implemented, a "rogue" task/request may bypass the locking mechanism and force a change to the underlying data. Also, if a lock is based on a session, a user may have multiple tabs open in the same browser, using the same session state (via shared cookies) Solving this problem will reduce the chance that when a person does post a form update, that there is any conflict, meaning fewer tears. *3)* Not knowing that data has changed on the server until you submit a form. Ideally there would be a means for someone viewing/editing a form to immediately be notified if data changes on the server, obsoleting the current form. This reduces the amount of wasted time is spent completing a form which is already known to be out of sync, and will need to be redone anyway (as long as problem 1 is solved; otherwise, there'll be data loss) *4)* Smarter form validation There are three types of missing validation: - the first is that the default widgets do not support even very simple client-side validation. For example, a text field might need to match a regular expression. - the second type is an ability to provide (in the model definition) arbitrary javascript which can be executed client-side to provide richer realtime validation during data entry. - the third type involves effectively providing provisional form data to the server, and having Django validate() the form content without actually saving the result. This would allow (for example) inter-field dependencies to be evaluated without any custom code, providing near-realtime feedback to the user that their form is invalid *The solutions* This is based on a day or so's experimentation, and I very much welcome any feedback, both in terms of the usefulness of solving these problems in general, as well as suggestion on better ways to solve the problems, before I go too far down any rabbit holes. *Enhanced forms* - when rendering a form (using e.g. as_p()), alongside the normal INPUT DOM elements, include additional hidden fields which store a copy of each form field's initial value. - when a form is submitted, compare these hidden values against the current value in the database. If any of these do not match, the clean() method can raise a ValidationError, allowing the user to know what has happened, and that they will need to reload the form and try again, with the new stored values. This solution is minimally invasive. As well as modifying as_p() and friends, a django template tag can also be exposed for those users who are rendering their forms in a different way. Note that there is no reliance on additional attributed in the models: the CAS-like checking performed is explicitly on the rendered form fields; it does not matter if other model fields' values have changed, as someone editing the form can neither see these field values, nor will their POSTing modify these other fields' values. (I have implemented the above already, for generic model forms using a single model) *Locking* - provide a mixin which can be used on selected models. When used, a view (usually some sort of form view) can attempt to lock() the model. If successful (because it's not currently locked to someone else), only they can perform writes to the model, until the lock expires. - if the lock has expired, anyone (including the user who took out an expired lock) may overate on the model instance. - the lock can be configu
