Bob,
On 24/08/11 12:10, bob the builder wrote:
i have a website as a backend.
i want to have a django site on another server to connect to the
backend website for login in and doing other stuff.
basically mysite would take login credentials, pass to other site and
receive a cookie.
then my site would send a request for data, and allow the user to
manipulate it on my server before sending it back to backend.
how would i go about this?
You're going to run into timeout, load, and other general
performance/architecture issues if you try to do this all in a standard
web response. Probably what you should do is store credentials in a
table, and log requests/events for third-party processing in a database
table, too, something like this:
RetrievalRequests
------------------------
RetrievalRequestID Username Password
15 encrypted_username1 encrypted_passwd1
16 encrypted_username2 encrypted_passwd2
RetrievalResults
--------------------
RetrievalResultID RetrievalRequestID State
11 14
FAILED
12 15
SUCCESS
13 16
RETRY
Then you need an external job, like a cron job or daemon, to
periodically look for new/retry requests, and process them.
The cron job should probably use something like oath + a proper API, or
failing that, something like mechanise[1] to interface with the
third-party site.
Note that:
a) You want to do this with a database setup that supports row-level
locking and transactions, for performance and data integrity.
b) No more information is shared than necessary in the above tables.
The backend process doesn't know anything about which web user requested
the data, for instance. Cookies etc. can stay in tables that only the
backend knows about. You could even use some sort of asymmetric key
system so that the front end can encode passwords for the third-party
site, but only the backend can decode them again. This would make it a
fair bit harder for site hackers to get hold of users' passwords -- from
basic web attacks like sql injection, at least.
Depending on the "manipulation", you could (and probably should) do your
data manipulation in a similar way:
ManipulationRequests
Id ObjectID ManipType
1023 543412 TRANSMOGRIFY
1024 345354 SCALE2X
ManipulationResults
...
Since they only need to access the DB, these background jobs can run on
a separate server from your site, or even multiple servers, if the load
ever demands it. If these were video encoding or 3d rendering requests,
for instance, dedicated machines optimised for those tasks could work
separately from your very differently configured webservers.
Finally, your site itself can just quickly check the table for
successful results and process responses to the client accordingly (send
the finished file, or whatever). You could even do this checking
through a thin server-side layer over AJAX to reduce load as much as
possible. But don't let AJAX directly access your DB or anything crazy
like that.
For the manipulation, if you have multiple manipulations for one object,
"scale; add blur; add sepia", then you can queue these as separate, more
granular requests, and use that directly for granular progress reports
to the user. i.e., if 1 retrieval and 4 manipulations were queued, and
the retrieval and 1st manipulation is done, then you're at stage 3 out
of 5 = 6/10 = 60% of your ajax progress bar.
This might sound like a lot of work, but it's a much better setup than
trying to do it all during a django request, and shouldn't actually take
much more work, since python can run as happily standalone as in a web
environment. For django compatibility, a few imports will give you
access to all the same classes/methods.
All that said, I'm just pulling this out of a non-pristine area, so
someone else might have a better suggestion based on working code :)
Hope this helps,
--
Lee
--
You received this message because you are subscribed to the Google Groups "Python
Ireland" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/pythonireland?hl=en.
