[pylons-discuss] How to get route_url() and static_url() to work from a Celery process?
Hi,
When I call certain Pyramid methods like route_url() or static_url() from a
Celery task they return the wrong URLs because they don't have the hostname and
port that'd normally come from the HTTP request's Host header.
I've found that I can fix it by setting an HTTP_HOST envvar in my OS
environment and then copying that envvar into the WSGI environment:
`request.environ["HTTP_HOST"] = os.environ["HTTP_HOST"]`. This seems to work
but I'm not sure if it's the right/best thing to do?
Details:
I run Celery like this:
celery -A lms.tasks.celery:app worker --loglevel=INFO
Link:
https://github.com/hypothesis/lms/blob/002afc26750f83faa5b1a7d7781817da9a2ba260/conf/supervisord-dev.conf#L20
Then in my celery.py file I set up an artificial Pyramid request like this:
from contextlib import contextmanager
from pyramid.scripting import prepare
from lms.app import create_app
lms = create_app(None, **{})
@contextmanager
def request_context():
with prepare(registry=lms.registry) as env:
yield env["request"]
sender.app.request_context = request_context
Link:
https://github.com/hypothesis/lms/blob/002afc26750f83faa5b1a7d7781817da9a2ba260/lms/tasks/celery.py#L75-L95
Each Celery task then does this to get an artificial Pyramid request:
with app.request_context() as request:
...
Example:
https://github.com/hypothesis/lms/blob/002afc26750f83faa5b1a7d7781817da9a2ba260/lms/tasks/mailchimp.py#L20-L25
The problem is that if any code called by one of these Celery tasks uses
request.route_url() or request.static_url() it's getting URLs without the right
host and port.
One way I've found to fix this is to set an HTTP_HOST envvar in the OS
environment and then inject it into the WSGI environment like this:
@contextmanager
def request_context():
with prepare(registry=lms.registry) as env:
request = env["request"]
# Make Pyramid things like route_url() and static_url() use the
# right hostname and port when called by Celery tasks.
request.environ["HTTP_HOST"] = os.environ["HTTP_HOST"]
yield request
See https://github.com/hypothesis/lms/pull/5657/. This works but I'm not sure
whether it's the best/right thing to do?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/pylons-discuss/5b7c52a2-4cd0-4887-916d-87480feed124%40app.fastmail.com.
Re: [pylons-discuss] How to pass a complex validation object to a forbidden view?
> You’re free to subclass HTTPForbidden or just extend it. But if I subclass HTTPForbidden, how do I get an instance of my subclass to be passed to the forbidden view (as request.exception) instead of the standard HTTPForbidden class? I thought it was Pyramid rather than user code that constructs that HTTPForbidden object. > You can also > subclass Denied as you noted. The big question to me is whether you can > control the generation of that object to use the subclass and off the > top of my head I think it’s always done in user code right now if > you’re using the new SecurityPolicy interface but I could be wrong! Yeah, this looks to me like it would work. I'll give it a try > You > just have to program the forbidden view a little defensively or make a > separate one for your subclass so that you can still handle exceptions > generated from code you may not control. Yep, good tip -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/4663be4d-4334-4c7d-b24e-1101866dd571%40www.fastmail.com.
[pylons-discuss] How to pass a complex validation object to a forbidden view?
If a view raises an exception, say a custom ValidationError class, then that ValidationError object is set as request.exception when the exception view is called (if you have registered a matching custom exception view). On the other hand if a security policy denies access (returning a Denied object) then the forbidden view (again: if you've registered one) gets called with an HTTPForbidden object as request.exception. Is there a nice way to attach a custom error object to the forbidden view? For example by replacing the HTTPForbidden with a custom exception class, or by attaching an exception or dict to the HTTPForbidden? The Denied object that the security policy returned is available to the forbidden view as request.exception.result. The Denied class's only attribute is a msg string. But I suppose the security policy could just attach a custom attribute to the Denied object? Or would you implement a custom subclass of Denied? That's what ACLSecurityPolicy seems to do. Thanks! -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/bcf4aa9c-205d-4dc1-a684-20fa1f59a889%40www.fastmail.com.
[pylons-discuss] Can you use an enum for permissions?
Do permissions in Pyramid always have to be strings, or can you use an enum? Currently we use this enum-like class to store all our permission strings as namespaced constants: class Permission: class User: READ = "user:read" CREATE = "user:create" UPDATE = "user:update" class Admin: FOO = "admin:foo" This is just as an alternative to always duplicating the literal strings everywhere. It works nicely with autocomplete, allows PyLint to catch typos, etc. It *seems* to work fine if we use enums: from enum import Enum class Permission: class User(Enum): READ = "user:read" CREATE = "user:create" UPDATE = "user:update" class Admin(Enum): FOO = "admin:foo" The reason is just so that we can get some of the functionality of Enum, like being able to iterate over them. >From what I can tell from the Pyramid docs permissions are usually strings, >but I don't see any concrete reason why arbitrary values can't be used for >permissions. Built-in permissions like security.Authenticated are strings but >I don't think that will cause any problems if our own permissions are enums. This seems to be fine? -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/b87825cb-be39-4a51-adb4-7eb20788ecc1%40www.fastmail.com.
Re: [pylons-discuss] Replacing Freenode?
I can no longer access the #pyramid channel because the IRC client that I use (IRCCloud) has been banned from Freenode: https://twitter.com/IRCCloud/status/1404153550159159298 #python moved to Libera: https://nedbatchelder.com/blog/202106/goodbye_freenode.html -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/52b80bec-3bee-4489-88d1-66c8c81065e9%40www.fastmail.com.
Re: [pylons-discuss] get Pshell to recognize Ipython.
On Fri, 30 Apr 2021, at 11:22 AM, pzzcc wrote: > Hello , > > I am trying to use Ipython as a shell to pshell using the command : > > *pshell -l * > ** > but the pshell doesn't list ipython as a shell that I can use. I can't find > anything in the docs about this. > > https://docs.pylonsproject.org/projects/pyramid/en/latest/pscripts/pshell.html > > any heads up please or where to look ? Hey, I think https://github.com/Pylons/pyramid_ipython does it for you -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/0d5e9bd9-7910-4a02-87fd-4f99af14ebea%40www.fastmail.com.
[pylons-discuss] Hypothesis is hiring Pyramid / Python developers
https://web.hypothes.is/jobs/senior-backend-developer/ Fully-remote and open source work. If you like working remotely then Hypothesis is a great place to work. We have a great team, everyone is really nice, the organization is kind to its employees, and the work is interesting. The job will mainly involve helping with our (currently) four Pyramid apps: https://github.com/hypothesis/h https://github.com/hypothesis/lms https://github.com/hypothesis/via3 https://github.com/hypothesis/bouncer ... plus various supporting Pyramid extensions and libs. We're currently a team of five developers (three frontend and two backend) and are hiring a few more. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/0700fb1b-8596-45dd-b4b7-5e35697f71cb%40www.fastmail.com.
[pylons-discuss] "Header names must be latin1 string" AssertionError from WebTest
Hi,
I'm wondering if anyone has ideas for how to deal with a "Header names must be
latin1 string" AssertionError from WebTest that occurs when Pyramid apps are
run in Python 2 with `from __future__ import unicode_literals`? Or what people
think of my tween-based workaround below?
## Problem
We're in the process of migrating our Pyramid app from Python 2 to 3, and added
`from __future__ import unicode_literals` to all our Python files, and this
caused the following error from WebTest in Python 2:
AssertionError: Header names must be latin1 string (not Py2 unicode or Py3
bytes type).
Here's a full traceback in case anyone's interested, though I don't think it's
very enlightening:
https://gist.github.com/seanh/1e8bee4476eca4d7f29e4c8d62f01171
The AssertionError is raised whenever your app sets any response header. For
example this line would trigger it:
response.headers["Access-Control-Allow-Origin"] = "*"
Since we have `unicode_literals` those string literals are unicode strings in
Python 2, whereas before `unicode_literals` they would have been byte strings,
that's why adding `unicode_literals` triggered the AssertionError from WebTest.
In Python 3 no error occurs.
The reason WebTest has this AssertionError is that
https://www.python.org/dev/peps/pep- requires HTTP response headers to be
native strings - byte strings in Python 2 and unicode strings in Python 3.
Here's the WebTest issue and pull request that added the assert:
https://github.com/Pylons/webtest/issues/119
https://github.com/Pylons/webtest/pull/180
## Workarounds
b-prefixing the strings like response.headers[b"Access-Control-Allow-Origin"] =
b"*" would get rid of the AssertionError in Python 2 but cause the error to
appear if the tests were run in Python 3.
Wrapping the strings in str() like
response.headers[str("Access-Control-Allow-Origin")] = str("*") will fix it in
both Python 2 and 3, but requires you to find and str()-wrap every response
header string throughout your app.
Adding a tween that str()-wraps all response headers seems to be a good fix:
def encode_headers_tween_factory(handler, registry):
def encode_headers_tween(request):
resp = handler(request)
for key in resp.headers.keys():
values = resp.headers.getall(key)
del resp.headers[key]
for value in values:
resp.headers.add(str(key), str(value))
return resp
return encode_headers_tween
config.add_tween('h.tweens.encode_headers_tween_factory')
--
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to pylons-discuss+unsubscr...@googlegroups.com.
To post to this group, send email to pylons-discuss@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/pylons-discuss/1523971797.981866.1340966176.0AF0BC64%40webmail.messagingengine.com.
For more options, visit https://groups.google.com/d/optout.
