django_auth_ldap cannot get group search to work
Hi all, I am using django_auth_ldap to authenticate users. I want to limit them to members of a group 'MT' but I cannot get it to work even though it looks like I am doing things as http://packages.python.org/django-auth-ldap/ shows me :-( This is my relevant stuff from settings.py: import ldap from django_auth_ldap.config import LDAPSearch, GroupOfNamesType AUTH_LDAP_SERVER_URI = "ldap://192.168.0.1:389"; AUTH_LDAP_BIND_DN = "" AUTH_LDAP_BIND_PASSWORD = "" AUTH_LDAP_USER_SEARCH = LDAPSearch("ou=users,o=mapper", ldap.SCOPE_SUBTREE, "(uid=%(user)s)") AUTH_LDAP_USER_ATTR_MAP = {"first_name": "givenName", "last_name": "sn", "email": "mail"} AUTH_LDAP_GROUP_SEARCH = LDAPSearch("ou=groups,o=mapper", ldap.SCOPE_SUBTREE, "(objectClass=groupOfNames)") AUTH_LDAP_GROUP_TYPE = GroupOfNamesType() AUTH_LDAP_REQUIRE_GROUP = "cn=MT,ou=groups,o=mapper" AUTHENTICATION_BACKENDS = ( 'django_auth_ldap.backend.LDAPBackend', 'django.contrib.auth.backends.ModelBackend', ) The LDIF dump from the server is: dn: cn=MT,ou=groups,o=mapper changetype: add sambaGroupType: 2 sambaSID: S-1-5-21-3621989192-1070409864-1511258855-3051 gidNumber: 1025 displayName: MT equivalentToMe: uid=michel,ou=users,o=mapper objectClass: Top objectClass: posixGroup objectClass: groupOfNames objectClass: sambaGroupMapping member: uid=michel,ou=users,o=mapper cn: MT And the debug logging says this: DEBUG 2011-11-22 11:21:28,347 config 28037 140700755687168 search_s('ou=users,o=mapper', 2, '(uid=michel)') returned 1 objects: uid=michel,ou=users,o=mapper DEBUG 2011-11-22 11:21:28,350 backend 28037 140700755687168 uid=michel,ou=users,o=mapper is not a member of cn=MT,ou=groups,o=mapper DEBUG 2011-11-22 11:21:28,350 backend 28037 140700755687168 Authentication failed for michel Can anyone see what I am doing wrong here? Thanks, Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: urlencode in template gives unexpected result (for me :-)
Thanks Tom that clarifies a lot, learning every day. My filesystem is ext4, encoding is irrelevant here right? So, I guess the best thing to do is to convert my database into utf-8 using a method as described here: http://www.bothernomore.com/2008/12/16/character-encoding-hell/ That way I'm consistently using utf-8. Would this also be backwards compatible with my legacy app? I don't see it using any encoding specific. Thanks, Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: urlencode in template gives unexpected result (for me :-)
On Aug 24, 3:22 pm, Tom Evans wrote: > On Wed, Aug 24, 2011 at 1:47 PM, Michel30 wrote: > > Hi all, > > > I have written an application using Django 1.3 , apache2 and a mysql > > db. > > I'm using the db to store filepaths and filenames for legacy purposes > > while serving them to users with apache. > > > Now mysql is using latin-1 (with the filenames most likely stored in > > CP-1252) while Django uses utf-8. > > That is not going to fly. You will likely need to ensure you have a > consistent character encoding across your website, database and file > system. > > Cheers > > Tom Tom, that looks like it would be best, yes (this is my first exposure to encoding problems) I cannot change the filesystem or mysql encoding since the legacy application is still using it. I assumed that with utf-8 I would be good as it covers all(?) and I understood mysql translates itself from latin-1 to utf-8 and vice versa. As far as I can see this only hurts my hyperlinks, more specifically only file.filename so wouldn't translating only these work? -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
urlencode in template gives unexpected result (for me :-)
Hi all, I have written an application using Django 1.3 , apache2 and a mysql db. I'm using the db to store filepaths and filenames for legacy purposes while serving them to users with apache. Now mysql is using latin-1 (with the filenames most likely stored in CP-1252) while Django uses utf-8. I generate the links to the files thusly in my template: {{ file.filename }} This works until I have funky character, lets say File….pdf Then my hyperlink reads: File….pdf While Apache throws a 404 with: NotFound /path/File….pdf Obviously because it expects this link: File….pdf Any ideas how to fix this in the template? Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Weird hyperlink issue with Office and Django authentication
Yes I agree, but if I understand it right then the article suggests that the issue originates from the single sign-on mechanism, in this case the Django authentication backend. This should be smart enough to handle the multiple sessions. Anyway I had a hard time finding anything related on the web including a fix. So for future reference, here it is. On Aug 16, 2:35 pm, Konstantin Sushenko wrote: > as the article says, their sessions are independent of these used by > the web server. the cookies are not shared between office sessions and > web sessions. I think django has nothing to do with it. > > On Aug 16, 4:04 am, Michel30 wrote: > > > > > > > > > Hmm the workaround MS suggests actually works, so I have a > > workaround :-) > > > But understanding why Django's authentication backend doesn't support > > this would be good > > > On Aug 16, 9:10 am, Michel30 wrote: > > > > Addidtionally I found this kb article that I believe describes what is > > > going on:http://support.microsoft.com/kb/899927/en-us > > > > On Aug 16, 8:59 am, Michel30 wrote: > > > > > I have made a CMS-like application to store and retrieve documents > > > > using Django 1.3, mysql and Apache. > > > > > One requirement was that only authenticated users could use it and > > > > that after closing the browser the session should be expired. > > > > So, I implemented an LDAP authentication backend, set the > > > > "SESSION_EXPIRE_AT_BROWSER_CLOSE" to True and added an @login_required > > > > decorator to functions that need it. > > > > And it is working great, just like it should. > > > > > Until using one feature that involves clicking on a hyperlink that is > > > > supposed to open a document within the application in a browser. > > > > This again works perfectly with every browser (firefix, chrome, IE) in > > > > several flavors of versions. > > > > With working I mean the document view opens, and this is important, as > > > > long as the user was already logged in to the application he/she > > > > doesn't have to log in again. > > > > > It is working until trying this from any office (2000/2003) > > > > application. Then suddenly logging in is required every time > > > > regardless if an session exists or not. > > > > I did some investigating and found that clicking a hyperlink in an > > > > Office application triggers three requests, and it looks like the > > > > first one is missing the session cookie. > > > > Apache log for a hyperlink in Open office: > > > > > 172.16.3.51 - - [15/Aug/2011:11:27:29 +0200] "GET /cgi-bin/DocDB/ > > > > ShowDocument?docid=20916&version=8 HTTP/1.1" 200 3603 "-" "Mozilla/5.0 > > > > (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0" > > > > > and the same document from Word: > > > > > 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /cgi-bin/DocDB/ > > > > ShowDocument?docid=20916&version=8 HTTP/1.1" 302 496 "-" "Mozilla/4.0 > > > > (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR > > > > 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR > > > > 3.5.30729; .NET4.0C; .NET4.0E)" > > > > > 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /login/?next=/ > > > > cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 > > > > 3269 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/ > > > > 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR > > > > 3.5.30729; .NET4.0C; .NET4.0E)" > > > > > 172.16.3.51 - - [15/Aug/2011:11:26:10 +0200] "GET /login/?next=/ > > > > cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 > > > > 3278 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/ > > > > 5.0" > > > > > Note the different status '200 = OK' for OO and '302 Found' for Word. > > > > Also the length of the first returned object differ: 496 vs 3603. > > > > > I am not sure where to point the finger to, or how to proceed and > > > > resolve it... So I'd appreciate some help a lot. > > > > Thanks, > > > > Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Weird hyperlink issue with Office and Django authentication
Hmm the workaround MS suggests actually works, so I have a workaround :-) But understanding why Django's authentication backend doesn't support this would be good On Aug 16, 9:10 am, Michel30 wrote: > Addidtionally I found this kb article that I believe describes what is > going on:http://support.microsoft.com/kb/899927/en-us > > On Aug 16, 8:59 am, Michel30 wrote: > > > > > > > > > I have made a CMS-like application to store and retrieve documents > > using Django 1.3, mysql and Apache. > > > One requirement was that only authenticated users could use it and > > that after closing the browser the session should be expired. > > So, I implemented an LDAP authentication backend, set the > > "SESSION_EXPIRE_AT_BROWSER_CLOSE" to True and added an @login_required > > decorator to functions that need it. > > And it is working great, just like it should. > > > Until using one feature that involves clicking on a hyperlink that is > > supposed to open a document within the application in a browser. > > This again works perfectly with every browser (firefix, chrome, IE) in > > several flavors of versions. > > With working I mean the document view opens, and this is important, as > > long as the user was already logged in to the application he/she > > doesn't have to log in again. > > > It is working until trying this from any office (2000/2003) > > application. Then suddenly logging in is required every time > > regardless if an session exists or not. > > I did some investigating and found that clicking a hyperlink in an > > Office application triggers three requests, and it looks like the > > first one is missing the session cookie. > > Apache log for a hyperlink in Open office: > > > 172.16.3.51 - - [15/Aug/2011:11:27:29 +0200] "GET /cgi-bin/DocDB/ > > ShowDocument?docid=20916&version=8 HTTP/1.1" 200 3603 "-" "Mozilla/5.0 > > (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0" > > > and the same document from Word: > > > 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /cgi-bin/DocDB/ > > ShowDocument?docid=20916&version=8 HTTP/1.1" 302 496 "-" "Mozilla/4.0 > > (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR > > 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR > > 3.5.30729; .NET4.0C; .NET4.0E)" > > > 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /login/?next=/ > > cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 > > 3269 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/ > > 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR > > 3.5.30729; .NET4.0C; .NET4.0E)" > > > 172.16.3.51 - - [15/Aug/2011:11:26:10 +0200] "GET /login/?next=/ > > cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 > > 3278 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/ > > 5.0" > > > Note the different status '200 = OK' for OO and '302 Found' for Word. > > Also the length of the first returned object differ: 496 vs 3603. > > > I am not sure where to point the finger to, or how to proceed and > > resolve it... So I'd appreciate some help a lot. > > Thanks, > > Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Weird hyperlink issue with Office and Django authentication
Addidtionally I found this kb article that I believe describes what is going on: http://support.microsoft.com/kb/899927/en-us On Aug 16, 8:59 am, Michel30 wrote: > I have made a CMS-like application to store and retrieve documents > using Django 1.3, mysql and Apache. > > One requirement was that only authenticated users could use it and > that after closing the browser the session should be expired. > So, I implemented an LDAP authentication backend, set the > "SESSION_EXPIRE_AT_BROWSER_CLOSE" to True and added an @login_required > decorator to functions that need it. > And it is working great, just like it should. > > Until using one feature that involves clicking on a hyperlink that is > supposed to open a document within the application in a browser. > This again works perfectly with every browser (firefix, chrome, IE) in > several flavors of versions. > With working I mean the document view opens, and this is important, as > long as the user was already logged in to the application he/she > doesn't have to log in again. > > It is working until trying this from any office (2000/2003) > application. Then suddenly logging in is required every time > regardless if an session exists or not. > I did some investigating and found that clicking a hyperlink in an > Office application triggers three requests, and it looks like the > first one is missing the session cookie. > Apache log for a hyperlink in Open office: > > 172.16.3.51 - - [15/Aug/2011:11:27:29 +0200] "GET /cgi-bin/DocDB/ > ShowDocument?docid=20916&version=8 HTTP/1.1" 200 3603 "-" "Mozilla/5.0 > (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0" > > and the same document from Word: > > 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /cgi-bin/DocDB/ > ShowDocument?docid=20916&version=8 HTTP/1.1" 302 496 "-" "Mozilla/4.0 > (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR > 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR > 3.5.30729; .NET4.0C; .NET4.0E)" > > 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /login/?next=/ > cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 > 3269 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/ > 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR > 3.5.30729; .NET4.0C; .NET4.0E)" > > 172.16.3.51 - - [15/Aug/2011:11:26:10 +0200] "GET /login/?next=/ > cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 > 3278 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/ > 5.0" > > Note the different status '200 = OK' for OO and '302 Found' for Word. > Also the length of the first returned object differ: 496 vs 3603. > > I am not sure where to point the finger to, or how to proceed and > resolve it... So I'd appreciate some help a lot. > Thanks, > Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Weird hyperlink issue with Office and Django authentication
I have made a CMS-like application to store and retrieve documents using Django 1.3, mysql and Apache. One requirement was that only authenticated users could use it and that after closing the browser the session should be expired. So, I implemented an LDAP authentication backend, set the "SESSION_EXPIRE_AT_BROWSER_CLOSE" to True and added an @login_required decorator to functions that need it. And it is working great, just like it should. Until using one feature that involves clicking on a hyperlink that is supposed to open a document within the application in a browser. This again works perfectly with every browser (firefix, chrome, IE) in several flavors of versions. With working I mean the document view opens, and this is important, as long as the user was already logged in to the application he/she doesn't have to log in again. It is working until trying this from any office (2000/2003) application. Then suddenly logging in is required every time regardless if an session exists or not. I did some investigating and found that clicking a hyperlink in an Office application triggers three requests, and it looks like the first one is missing the session cookie. Apache log for a hyperlink in Open office: 172.16.3.51 - - [15/Aug/2011:11:27:29 +0200] "GET /cgi-bin/DocDB/ ShowDocument?docid=20916&version=8 HTTP/1.1" 200 3603 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0" and the same document from Word: 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /cgi-bin/DocDB/ ShowDocument?docid=20916&version=8 HTTP/1.1" 302 496 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)" 172.16.3.51 - - [15/Aug/2011:11:26:09 +0200] "GET /login/?next=/ cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 3269 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/ 4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)" 172.16.3.51 - - [15/Aug/2011:11:26:10 +0200] "GET /login/?next=/ cgi-bin/DocDB/ShowDocument%3Fdocid%3D20916%26version%3D8 HTTP/1.1" 200 3278 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/ 5.0" Note the different status '200 = OK' for OO and '302 Found' for Word. Also the length of the first returned object differ: 496 vs 3603. I am not sure where to point the finger to, or how to proceed and resolve it... So I'd appreciate some help a lot. Thanks, Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: speeding up iterating over query set
I have tried and I think I have it mostly working: it returns ALL unique docid's. What is left is that my code is part of a search function. Originally I got normalized keywords from a user and used those Q-objects to look for keywords in a selected set of columns. I still have to figure out how to get that into the SQL part.. On Jul 12, 3:29 pm, bruno desthuilliers wrote: > On Jul 12, 12:26 pm, Michel30 wrote: > > > Hi guys, > > > I've been trying your suggestions but I'm afraid I'm stretching the > > limits of my Python/Django abilities ;-) > > > Bruno got it right: what I want is a queryset of "model" with distinct > > docid having the highest version number, sorted by revisiondate. > > (snip) > > My code does this, but the loop that selects the distinct docid's is > > what makes it terribly slow... > > Then why don't you just try the solution(s) I posted ? -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: speeding up iterating over query set
Hi guys, I've been trying your suggestions but I'm afraid I'm stretching the limits of my Python/Django abilities ;-) Bruno got it right: what I want is a queryset of "model" with distinct docid having the highest version number, sorted by revisiondate. If have the following result of my found_entries = model.objects.filter((Q- objects),obsolete=0).order_by('-version','docid') : ++---+-+--+ | pk | docid | version | revisiondate | ++---+-+--+ | 1 | 1 | 1 | 2000-02-10 | | 2 | 2 | 1 | 2000-02-11 | | 3 | 3 | 1 | 2000-02-12 | | 4 | 3 | 3 | 2000-02-13 | | 5 | 2 | 3 | 2000-02-14 | | 6 | 1 | 3 | 2000-02-15 | ++---+-+--+ Then I want to retrieve only these results, sorted on revisiondate: ++---+-+--+ | pk | docid | version | revisiondate | ++---+-+--+ | 6 | 1 | 3 | 2000-02-15 | | 5 | 2 | 3 | 2000-02-14 | | 4 | 3 | 3 | 2000-02-13 | ++---+-+--+ My code does this, but the loop that selects the distinct docid's is what makes it terribly slow... Hope this clarifies it. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: speeding up iterating over query set
Hi guys, I've been trying your suggestions but I'm afraid I'm stretching the limits of my Python/Django abilities ;-) Bruno got it right: what I want is a queryset of "model" with distinct docid having the highest version number, sorted by revisiondate. If have the following result of my found_entries = model.objects.filter((Q- objects),obsolete=0).order_by('-version','docid') : ++---+-+--+ | pk | docid | version | revisiondate | ++---+-+--+ | 1 | 1 | 1 | 2000-02-10 | | 2 | 2 | 1 | 2000-02-11 | | 3 | 3 | 1 | 2000-02-12 | | 4 | 3 | 3 | 2000-02-13 | | 5 | 2 | 3 | 2000-02-14 | | 6 | 1 | 3 | 2000-02-15 | ++---+-+--+ Then I want to retrieve only these results, sorted on revisiondate: ++---+-+--+ | pk | docid | version | revisiondate | ++---+-+--+ | 6 | 1 | 3 | 2000-02-15 | | 5 | 2 | 3 | 2000-02-14 | | 4 | 3 | 3 | 2000-02-13 | ++---+-+--+ My code does this, but the loop that selects the distinct docid's is what makes it terribly slow... Hope this clarifies it. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: speeding up iterating over query set
Hi guys, I've been trying your suggestions but I'm afraid I'm stretching the limits of my Python/Django abilities ;-) Bruno got it right: what I want is a queryset of "model" with distinct docid having the highest version number, sorted by revisiondate. If have the following result of my found_entries = model.objects.filter((Q- objects),obsolete=0).order_by('-version','docid') : ++---+-+--+ | pk | docid | version | revisiondate | ++---+-+--+ | 1 | 1 | 1 | 2000-02-10 | | 2 | 2 | 1 | 2000-02-11 | | 3 | 3 | 1 | 2000-02-12 | | 4 | 3 | 3 | 2000-02-13 | | 5 | 2 | 3 | 2000-02-14 | | 6 | 1 | 3 | 2000-02-15 | ++---+-+--+ Then I want to retrieve only these results, sorted on revisiondate: ++---+-+--+ | pk | docid | version | revisiondate | ++---+-+--+ | 6 | 1 | 3 | 2000-02-15 | | 5 | 2 | 3 | 2000-02-14 | | 4 | 3 | 3 | 2000-02-13 | ++---+-+--+ My code does this, but the loop that selects the distinct docid's is what makes it terribly slow... Hope this clarifies it. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: speeding up iterating over query set
I didn't try this approach yet, I'll give it a go and let you know my mileage. Thanks On Jul 11, 4:06 pm, "Cal Leeming [Simplicity Media Ltd]" wrote: > On Mon, Jul 11, 2011 at 3:04 PM, Cal Leeming [Simplicity Media Ltd] < > > > > cal.leem...@simplicitymedialtd.co.uk> wrote: > > > On Mon, Jul 11, 2011 at 2:57 PM, Michel30 wrote: > > >> Hi all, > > >> I have a basic search function that uses Q objects. > >> After profiling it I found that the actual (mysql) database query > >> finishes in fractions of seconds but the iterating after this can take > >> up to 50 seconds per 10.000 results. > > >> I have been trying to speed it up but I have had not much results.. > > >> My query is this one: > > >> found_entries = model.objects.filter((Q-objects), > >> obsolete=0).order_by('-version','docid') > > >> So far so good, but then I need a dictionary to retrieve only unique > >> 'documentid's'. > > > You could do: > > > # grab all results > > _res = > > model.objects.filter((Q-objects),obsolete=0).order_by('-version','docid').values() > > # re-map them into (id, obj) > > _res = map(lambda x: [x.docid, x], _res) > > # wrap in a dict(), which uses index position 0 as the key, and index > > position 1 as the value > > _res = dict(_res) > > Just tested the same principle, and it seems to work fine. It uses last > object found as the final choice if dups are found. > > >>> _res = [ [1,2], [1,3], [2,4], [2,5] ] > >>> map(lambda x: [x[0], x[1]], _res) > > [[1, 2], [1, 3], [2, 4], [2, 5]]>>> map(lambda x: [x[0], x[1]], _res) > > [[1, 2], [1, 3], [2, 4], [2, 5]] > > >>> dict(map(lambda x: [x[0], x[1]], _res)) > {1: 3, 2: 5} > > > Let me know if this give you the results you need. > > >> rev_dict = {} > > >> This is the part that hurts: > > >> for d in found_entries: > >> rev_dict[d.documentid] = d > > >> And then some more sorting and filtering: > > >> filtered_entries = rev_dict.values() > >> filtered_entries.sort(key=lambda d: d.revisiondate, reverse=True) > > >> Does anyone have some better ideas to achieve this? > > >> Thanks > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Django users" group. > >> To post to this group, send email to django-users@googlegroups.com. > >> To unsubscribe from this group, send email to > >> django-users+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/django-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
speeding up iterating over query set
Hi all, I have a basic search function that uses Q objects. After profiling it I found that the actual (mysql) database query finishes in fractions of seconds but the iterating after this can take up to 50 seconds per 10.000 results. I have been trying to speed it up but I have had not much results.. My query is this one: found_entries = model.objects.filter((Q-objects), obsolete=0).order_by('-version','docid') So far so good, but then I need a dictionary to retrieve only unique 'documentid's'. rev_dict = {} This is the part that hurts: for d in found_entries: rev_dict[d.documentid] = d And then some more sorting and filtering: filtered_entries = rev_dict.values() filtered_entries.sort(key=lambda d: d.revisiondate, reverse=True) Does anyone have some better ideas to achieve this? Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Stuck at filtering/slicing
hmm had a send accident.. I tried Documentrevision.objects.values('documentid').annotate(Max('versionnumber')).filter and it retrieves a list of values, but I need the objects to use later on On May 25, 2:40 pm, Michel30 wrote: > I've tried annotate before but I ran into the issue that it returns a > list of values I believe? > I tried: > > On May 25, 1:40 pm, Jani Tiainen wrote: > > > On Wed, 2011-05-25 at 02:59 -0700, Michel30 wrote: > > > Hello all, > > > > I have the following model: > > > > class Documentrevision(models.Model): > > > docrevid = models.AutoField(primary_key=True, > > > db_column='DocRevID') > > > documentid = models.ForeignKey(Document, db_column='DocumentID') > > > submitterid = models.ForeignKey('Author', db_column='SubmitterID') > > > documenttitle = models.CharField(max_length=765, > > > db_column='DocumentTitle') > > > publicationinfo = models.TextField(db_column='PublicationInfo', > > > blank=True) > > > versionnumber = models.IntegerField(db_column='VersionNumber') > > > abstract = models.TextField(db_column='Abstract', blank=True) > > > revisiondate = models.DateTimeField(null=True, > > > db_column='RevisionDate', blank=True) > > > timestamp = models.DateTimeField(db_column='TimeStamp') > > > obsolete = models.IntegerField(null=True, db_column='Obsolete', > > > blank=True) > > > keywords = models.CharField(max_length=720, db_column='Keywords', > > > blank=True) > > > note = models.TextField(db_column='Note', blank=True) > > > demanaged = models.IntegerField(null=True, db_column='Demanaged', > > > blank=True) > > > > Now I want to retrieve all entries that match obsolete=0, then order > > > on revisiondate and docrevid and last get only the highest > > > versionnumber for each group of documentid. > > > > I got this far: > > > found_entries = > > > Documentrevision.objects.filter(obsolete=0).order_by('-revisiondate','- > > > docrevid') > > > > This gives me all versionnumbers though: I've been trying several ways > > > to slice, group , do for-loops and can't get it to work.. > > > Any ideas are greatly appreciated :-) > > > Thanks > > > DocumentRevision.objects.values('documentid').annotate(Max('docrevid)').filter(...) > > > Should do the trick. > > > -- > > > Jani Tiainen > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Stuck at filtering/slicing
I've tried annotate before but I ran into the issue that it returns a list of values I believe? I tried: On May 25, 1:40 pm, Jani Tiainen wrote: > On Wed, 2011-05-25 at 02:59 -0700, Michel30 wrote: > > Hello all, > > > I have the following model: > > > class Documentrevision(models.Model): > > docrevid = models.AutoField(primary_key=True, > > db_column='DocRevID') > > documentid = models.ForeignKey(Document, db_column='DocumentID') > > submitterid = models.ForeignKey('Author', db_column='SubmitterID') > > documenttitle = models.CharField(max_length=765, > > db_column='DocumentTitle') > > publicationinfo = models.TextField(db_column='PublicationInfo', > > blank=True) > > versionnumber = models.IntegerField(db_column='VersionNumber') > > abstract = models.TextField(db_column='Abstract', blank=True) > > revisiondate = models.DateTimeField(null=True, > > db_column='RevisionDate', blank=True) > > timestamp = models.DateTimeField(db_column='TimeStamp') > > obsolete = models.IntegerField(null=True, db_column='Obsolete', > > blank=True) > > keywords = models.CharField(max_length=720, db_column='Keywords', > > blank=True) > > note = models.TextField(db_column='Note', blank=True) > > demanaged = models.IntegerField(null=True, db_column='Demanaged', > > blank=True) > > > Now I want to retrieve all entries that match obsolete=0, then order > > on revisiondate and docrevid and last get only the highest > > versionnumber for each group of documentid. > > > I got this far: > > found_entries = > > Documentrevision.objects.filter(obsolete=0).order_by('-revisiondate','- > > docrevid') > > > This gives me all versionnumbers though: I've been trying several ways > > to slice, group , do for-loops and can't get it to work.. > > Any ideas are greatly appreciated :-) > > Thanks > > DocumentRevision.objects.values('documentid').annotate(Max('docrevid)').filter(...) > > Should do the trick. > > -- > > Jani Tiainen -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Stuck at filtering/slicing
Hello all, I have the following model: class Documentrevision(models.Model): docrevid = models.AutoField(primary_key=True, db_column='DocRevID') documentid = models.ForeignKey(Document, db_column='DocumentID') submitterid = models.ForeignKey('Author', db_column='SubmitterID') documenttitle = models.CharField(max_length=765, db_column='DocumentTitle') publicationinfo = models.TextField(db_column='PublicationInfo', blank=True) versionnumber = models.IntegerField(db_column='VersionNumber') abstract = models.TextField(db_column='Abstract', blank=True) revisiondate = models.DateTimeField(null=True, db_column='RevisionDate', blank=True) timestamp = models.DateTimeField(db_column='TimeStamp') obsolete = models.IntegerField(null=True, db_column='Obsolete', blank=True) keywords = models.CharField(max_length=720, db_column='Keywords', blank=True) note = models.TextField(db_column='Note', blank=True) demanaged = models.IntegerField(null=True, db_column='Demanaged', blank=True) Now I want to retrieve all entries that match obsolete=0, then order on revisiondate and docrevid and last get only the highest versionnumber for each group of documentid. I got this far: found_entries = Documentrevision.objects.filter(obsolete=0).order_by('-revisiondate','- docrevid') This gives me all versionnumbers though: I've been trying several ways to slice, group , do for-loops and can't get it to work.. Any ideas are greatly appreciated :-) Thanks -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: retrieving current user in view
This is the traceback: Traceback: File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/ base.py" in get_response 111. response = callback(request, *callback_args, **callback_kwargs) File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/ decorators.py" in _wrapped_view 23. return view_func(request, *args, **kwargs) File "/home/ast1/workspace/CMT/cmt/views.py" in home 46. userID = retrieveUserID(firstname, lastname) File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/ decorators.py" in _wrapped_view 22. if test_func(request.user): Exception Type: AttributeError at / Exception Value: 'unicode' object has no attribute 'user' I also found my error using your pointers. I was passing the user's first and last name to another view that uses it in a function that has the @login_required decorator. That obviously doesn't work but I can remove the decorator there to fix my issue, thanks for the help guy's. Concerning my other question, the way I do it now is like this: Template: {% csrf_token %} {{ form }} View: class SimpleFileForm(forms.Form): upload_file = forms.FileField(label='', widget=forms.FileInput, required=True) This gives me a file field, a browse button and a submit button. What I'd like to have is a submit button only, click that opens a file dialog. Any ideas? Thanks, On 18 mei, 18:42, Oscar Carballal wrote: > As Daniel said, I dont' think also that the entire view is that, but > if it helps, I'll let you a piece of my code for a form (gpl, no > problem if you copy it). > > @permission_required('spaces.add_space') > def create_space(request): > > """ > Create new spaces. In this view the author field is automatically filled > so we can't use a generic view. > """ > space = Space() > form = SpaceForm(request.POST or None, request.FILES or None, > instance=space) > > if request.POST: > form_uncommited = form.save(commit=False) > form_uncommited.author = request.user > if form.is_valid(): > form_uncommited.save() > # We add the created spaces to the user allowed spaces > space = get_object_or_404(Space, name=form_uncommited.name) > request.user.profile.spaces.add(space) > return redirect('/spaces/' + space.url) > > return render_to_response('spaces/space_add.html', > {'form': form}, > context_instance=RequestContext(request)) > > 2011/5/18 Daniel Roseman : > > > On Wednesday, May 18, 2011 2:16:00 PM UTC+1,Michel30wrote: > > >> Hey all, > > >> I have a Django 1.3 app that retrieves user credentials from LDAP. > >> Most views require the user to be authenticated so I use the > >> @login_required decorator. > > >> Now, in a form a user can upload a document using a form: > > >> {% > >> csrf_token %} > >> {{ form }} > >> > >> > > >> I want to log the user's first and lastname who submitted the file in > >> a model. So, in my view I've tried a number of solutions but all came > >> up with various errors either related to the @login_required decorator > >> or complaining that no user exists in the POST object. > > >> This is the latest attempt I have in my view: > > >> def home(request): > >> form = SimpleFileForm() > >> if request.method == 'POST': > >> if 'upload_file' in request.FILES: > >> upload_file = request.FILES['upload_file'] > >> filename = request.FILES['upload_file'].name > >> user = request.user > >> firstname = user.first_name > >> lastname = user.last_name > >> etc... > > >> It throws this error: 'unicode' object has no attribute 'user' > > >> Does someone know a good way how to do this? (Any suggestions on how > >> to get rid of the file textfield / browse button and only leave a > >> submit button to open a select dialog are also very much appreciated) > > >> Thanks, > >> Michel > > > That is the correct way to do it. I don't think that is the real code you're > > running, because `request` is clearly not a unicode object in the line > > beginning with `filename`, and yet it is one line later. > > The full traceback would help, as would the actual code. > > I don't know what you mean by "a submit button to open a select dialog". > > -- > > DR. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Django users" group. > > To post to this group, send email to django-users@googlegroups.com. > > To unsubscribe from this group, send email to > > django-users+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/django-users?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, sen
retrieving current user in view
Hey all, I have a Django 1.3 app that retrieves user credentials from LDAP. Most views require the user to be authenticated so I use the @login_required decorator. Now, in a form a user can upload a document using a form: {% csrf_token %} {{ form }} I want to log the user's first and lastname who submitted the file in a model. So, in my view I've tried a number of solutions but all came up with various errors either related to the @login_required decorator or complaining that no user exists in the POST object. This is the latest attempt I have in my view: def home(request): form = SimpleFileForm() if request.method == 'POST': if 'upload_file' in request.FILES: upload_file = request.FILES['upload_file'] filename = request.FILES['upload_file'].name user = request.user firstname = user.first_name lastname = user.last_name etc... It throws this error: 'unicode' object has no attribute 'user' Does someone know a good way how to do this? (Any suggestions on how to get rid of the file textfield / browse button and only leave a submit button to open a select dialog are also very much appreciated) Thanks, Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: outer joins, raw sql or one-to-many?
wow this works :-) I think I mixed up the tables before: submitter = models.ForeignKey(Author, db_column='SubmitterID') vs submitter = models.ForeignKey(Author, db_column='authorid') the latter doesn't work obviously.. Thanks all, I think this will help me on my way! On May 10, 9:38 am, "Szabo, Patrick \(LNG-VIE\)" wrote: > The lather should work... > > . . . . . . . . . . . . . . . . . . . . . . . . . . > Patrick Szabo > XSLT Developer > LexisNexis > Marxergasse 25, 1030 Wien > > mailto:patrick.sz...@lexisnexis.at > Tel.: +43 (1) 534 52 - 1573 > Fax: +43 (1) 534 52 - 146 > > -Ursprüngliche Nachricht- > > Von: django-users@googlegroups.com [mailto:django-users@googlegroups.com] Im > Auftrag von Kenneth Gonsalves > Gesendet: Dienstag, 10. Mai 2011 09:34 > An: django-users@googlegroups.com > Betreff: Re: outer joins, raw sql or one-to-many? > > On Tue, 2011-05-10 at 00:26 -0700, Michel30 wrote: > > {{ some_thing.firstname }} > > maybe > {{ some_thing.submitter.firstname }} > -- > regards > KGhttp://lawgon.livejournal.com > Coimbatore LUG roxhttp://ilugcbe.techstud.org/ > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-users@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/django-users?hl=en. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: outer joins, raw sql or one-to-many?
Oops slip of the mouse there :-) I've tried setting up the relations before like my previous post. But then when I want to fetch data with, for example: some_thing = Documentrevision.objects.filter(some_filter criteria) and then use it in my form like this: {{ some_thing.firstname }} There is no data while I expected it to follow the relation back to the Author table.. On May 10, 9:18 am, Michel30 wrote: > Hi Kenneth, > > I tried that before using: > submitterid = models.ForeignKey('Author', to_field='authorid') > > On May 10, 9:07 am, Kenneth Gonsalves wrote: > > > On Mon, 2011-05-09 at 23:56 -0700, Michel30 wrote: > > > Now, for every documentid I retrieve I want to find it's firstname and > > > lastname. Authorid relates to submitterid in this case. > > > submitter should be foreign key to author - then all your problems will > > go away. > > -- > > regards > > KGhttp://lawgon.livejournal.com > > Coimbatore LUG roxhttp://ilugcbe.techstud.org/ > > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: outer joins, raw sql or one-to-many?
Hi Kenneth, I tried that before using: submitterid = models.ForeignKey('Author', to_field='authorid') On May 10, 9:07 am, Kenneth Gonsalves wrote: > On Mon, 2011-05-09 at 23:56 -0700, Michel30 wrote: > > Now, for every documentid I retrieve I want to find it's firstname and > > lastname. Authorid relates to submitterid in this case. > > submitter should be foreign key to author - then all your problems will > go away. > -- > regards > KGhttp://lawgon.livejournal.com > Coimbatore LUG roxhttp://ilugcbe.techstud.org/ -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
outer joins, raw sql or one-to-many?
Hey all, I'm having trouble wrapping my head around querying multiple tables with relations between them. I'm writing an (cms-like) app that will use an existing mysql database. I modeled that, got my code up and running and am now in the process of trying to retrieve and submit data to it. Here is an example of my model: class Author(models.Model): authorid = models.IntegerField(primary_key=True, db_column='AuthorID') firstname = models.CharField(max_length=96, db_column='FirstName') lastname = models.CharField(max_length=96, db_column='LastName') class Meta: db_table = u'Author' def __unicode__(self): return self.middleinitials class Documentrevision(models.Model): docrevid = models.IntegerField(primary_key=True, db_column='DocRevID') documentid = models.IntegerField(db_column='DocumentID') submitterid = models.IntegerField(db_column='SubmitterID') ... class Meta: db_table = u'DocumentRevision' def __unicode__(self): return self.documenttitle Now, for every documentid I retrieve I want to find it's firstname and lastname. Authorid relates to submitterid in this case. Can someone help me on my way on how to accomplish this? Many thanks, Michel -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: escaping metacharacter in url pattern
That is just terrific Raúl, thanks a lot! code I ended up using: url(r'^cgi-bin/DocDB/ ShowDocument','docDB.views.retrieveDocumentVersion') documentid = int(request.GET.get('docid')) version = request.GET.get('version', '') On Apr 21, 4:16 pm, Raúl Cumplido wrote: > sorry copy&paste error: > > *u*rl(r'^cgi-bin/DocDB/ShowDocument\$', > 'docDB.views.retrieveDocumentVersion'), > > 2011/4/21 Raúl Cumplido > > > > > Hi, > > > That data is not part of the path they are part of the querystring. It > > would be better to set your urls.py as: > > > rl(r'^cgi-bin/DocDB/ShowDocument\$', > > 'docDB.views.retrieveDocumentVersion'), > > > And retrieve values in your view as: > > > request.GET.get('docid', '') > > and > > request.GET.get('version', '') > > > Look at the documentation here: > > >http://docs.djangoproject.com/en/1.3/ref/request-response/#django.htt... > > > Raúl > > > On Thu, Apr 21, 2011 at 3:59 PM, Michel30 wrote: > > >> Hey guy's, > > >> I'm trying to replicate behaviour of a legacy CMS and stick it into a > >> new Django project. > > >> Here is an example of my url: > > >>http://hostname:port/cgi-bin/DocDB/ShowDocument?docid=19530&version=1 > > >> I want to filter the docid and version with a regex in a urlpattern to > >> use later in a function: > > >> url(r'^cgi-bin/DocDB/ShowDocument\?docid=(?P\d+)\? > >> version=(?P\d+)', 'docDB.views.retrieveDocumentVersion'), > > >> I've tried about every way of escaping the '? ' but can't get it to > >> work... > > >> Any ideas anyone? > > >> Thanks, > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Django users" group. > >> To post to this group, send email to django-users@googlegroups.com. > >> To unsubscribe from this group, send email to > >> django-users+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/django-users?hl=en. > > > -- > > Raúl Cumplido > > -- > Raúl Cumplido -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
escaping metacharacter in url pattern
Hey guy's, I'm trying to replicate behaviour of a legacy CMS and stick it into a new Django project. Here is an example of my url: http://hostname:port/cgi-bin/DocDB/ShowDocument?docid=19530&version=1 I want to filter the docid and version with a regex in a urlpattern to use later in a function: url(r'^cgi-bin/DocDB/ShowDocument\?docid=(?P\d+)\? version=(?P\d+)', 'docDB.views.retrieveDocumentVersion'), I've tried about every way of escaping the '? ' but can't get it to work... Any ideas anyone? Thanks, -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.