Re: [Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong -+- Reporter: Simon Chenard|Owner: (none) Type: Bug | Status: closed Component: Error reporting | Version: 1.10 Severity: Normal | Resolution: needsinfo Keywords: celery logging | Triage Stage: traceback | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Comment (by David Black): The issue seems to be caused by celery using billiard's ExceptionInfo and it providing its own _Frame and Traceback classes. (Seemingly related -> https://github.com/celery/billiard/issues/6). -- Ticket URL: <https://code.djangoproject.com/ticket/27543#comment:6> Django <https://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-updates+unsubscr...@googlegroups.com. To post to this group, send email to django-updates@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.1f9795b5ec1fde64d262e25d8c290d2c%40djangoproject.com. For more options, visit https://groups.google.com/d/optout.
Re: [Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong -+- Reporter: Simon Chenard|Owner: (none) Type: Bug | Status: closed Component: Error reporting | Version: 1.10 Severity: Normal | Resolution: needsinfo Keywords: celery logging | Triage Stage: traceback | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Comment (by Simon Chenard): In the end I couldn't completely understand the relationship between django and celery - but since it does only happens when celery is involved, it's most likely not a issue on Django's side. But for the record, if anyone else stumble on a similar case, what I did end up doing is sub-classing the AdminEmailHandler and use python's own traceback library, if the exception described earlier is raised. -- Ticket URL: <https://code.djangoproject.com/ticket/27543#comment:5> Django <https://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-updates+unsubscr...@googlegroups.com. To post to this group, send email to django-updates@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.88a2b80850f01bd0f41d4fd3cd16d4ee%40djangoproject.com. For more options, visit https://groups.google.com/d/optout.
Re: [Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong -+- Reporter: Simon Chenard|Owner: (none) Type: Bug | Status: closed Component: Error reporting | Version: 1.10 Severity: Normal | Resolution: needsinfo Keywords: celery logging | Triage Stage: traceback | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Changes (by Tim Graham): * status: new => closed * resolution: => needsinfo -- Ticket URL: <https://code.djangoproject.com/ticket/27543#comment:4> Django <https://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-updates+unsubscr...@googlegroups.com. To post to this group, send email to django-updates@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.e71e38535fb36d23368c54eaf13cb057%40djangoproject.com. For more options, visit https://groups.google.com/d/optout.
Re: [Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong -+- Reporter: Simon Chenard|Owner: (none) Type: Bug | Status: new Component: Error reporting | Version: 1.10 Severity: Normal | Resolution: Keywords: celery logging | Triage Stage: traceback | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Comment (by Tim Graham): Can you provide a way to reproduce the issue without celery to confirm it's a Django bug and not something that should be fixed in celery? -- Ticket URL: <https://code.djangoproject.com/ticket/27543#comment:3> Django <https://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-updates+unsubscr...@googlegroups.com. To post to this group, send email to django-updates@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.51834ad7fc8486735811bd8355956266%40djangoproject.com. For more options, visit https://groups.google.com/d/optout.
Re: [Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong -+- Reporter: Simon Chenard|Owner: (none) Type: Bug | Status: new Component: Error reporting | Version: 1.10 Severity: Normal | Resolution: Keywords: celery logging | Triage Stage: traceback | Unreviewed Has patch: 0| Needs documentation: 0 Needs tests: 0| Patch needs improvement: 0 Easy pickings: 0|UI/UX: 0 -+- Comment (by Simon Charette): That looks similar to #17699. -- Ticket URL: <https://code.djangoproject.com/ticket/27543#comment:2> Django <https://code.djangoproject.com/> The Web framework for perfectionists with deadlines. -- You received this message because you are subscribed to the Google Groups "Django updates" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-updates+unsubscr...@googlegroups.com. To post to this group, send email to django-updates@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-updates/067.9bb98909d2b8cbd8a82e1479a5730b14%40djangoproject.com. For more options, visit https://groups.google.com/d/optout.
Re: [Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and
celery error logging gone wrong
-+-
Reporter: Simon Chenard|Owner: (none)
Type: Bug | Status: new
Component: Error reporting | Version: 1.10
Severity: Normal | Resolution:
Keywords: celery logging | Triage Stage:
traceback | Unreviewed
Has patch: 0| Needs documentation: 0
Needs tests: 0| Patch needs improvement: 0
Easy pickings: 0|UI/UX: 0
-+-
Description changed by Simon Chenard:
Old description:
> When an error is raised inside a celery task, django crashes with the
> error described in the summary. This error is not raised anywhere else in
> the application, only in a celery task. Came across this situation trying
> to understand why I was not getting error emails for celery tasks.
> Triggered this exception by adding :
>
> CELERY_WORKER_HIJACK_ROOT_LOGGER = False
>
> Implying otherwise the error is not redirected to the AdminEmailHandler.
>
> Django version : 1.10.2
> Celery version : 4.0.0
>
> (stacktrace is below)
>
> The method "get_traceback_frame_variables" is the cause of this problem.
> By wrapping a try except around the code of the method and returning an
> empty array, I roughly get what I expect : an email describing the error
> with the adequate traceback.
>
> My logging setup :
> LOGGING = {
> 'version': 1,
> 'disable_existing_loggers': False,
> 'filters': {
> 'task_id': {
> '()': 'lib.logging.task_filter.TaskIDFilter'
> },
> 'require_debug_false': {
> '()': 'django.utils.log.RequireDebugFalse'
> }
> },
> 'formatters' : {
> 'task': {
> 'format': LOGGING_TASK_FORMAT
> },
> },
> 'handlers': {
> 'mail_admins': {
> 'level': 'ERROR',
> 'class': 'django.utils.log.AdminEmailHandler',
> 'include_html': True,
> 'filters': ['require_debug_false'], # commented or not, the
> error happens with DEBUG = True or False
> },
> },
> 'loggers': {
> '': {
> 'handlers': ['console', 'mail_admins'],
> 'level': 'INFO',
> 'propagate': True
> },
> }
> }
>
> Here is the stacktrace :
>
> Traceback (most recent call last):
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/celery/app/trace.py", line 382, in trace_task
> I, R, state, retval = on_error(task_request, exc, uuid)
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/celery/app/trace.py", line 324, in on_error
> task, request, eager=eager, call_errbacks=call_errbacks,
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/celery/app/trace.py", line 158, in handle_error_state
> call_errbacks=call_errbacks)
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/celery/app/trace.py", line 212, in handle_failure
> self._log_error(task, req, einfo)
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/celery/app/trace.py", line 242, in _log_error
> extra={'data': context})
> File "/usr/lib/python3.4/logging/__init__.py", line 1341, in log
> self._log(level, msg, args, **kwargs)
> File "/usr/lib/python3.4/logging/__init__.py", line 1409, in _log
> self.handle(record)
> File "/usr/lib/python3.4/logging/__init__.py", line 1419, in handle
> self.callHandlers(record)
> File "/usr/lib/python3.4/logging/__init__.py", line 1481, in
> callHandlers
> hdlr.handle(record)
> File "/usr/lib/python3.4/logging/__init__.py", line 853, in handle
> self.emit(record)
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/django/utils/log.py", line 119, in emit
> message = "%s\n\n%s" % (self.format(no_exc_record),
> reporter.get_traceback_text())
> File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
> packages/django/views/debug.py", line 325, in get_traceback_text
> c = Context(self.get_traceback_data(), autoescape=False,
> use_l10n=False)
> File "/home/simon/c
[Django] #27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and celery error logging gone wrong
#27543: AttributeError: '_Frame' object has no attribute 'f_back' / Django and
celery error logging gone wrong
-+-
Reporter: Simon | Owner: (none)
Chenard|
Type: Bug| Status: new
Component: Error |Version: 1.10
reporting | Keywords: celery logging
Severity: Normal | traceback
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 |Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-+-
When an error is raised inside a celery task, django crashes with the
error described in the summary. This error is not raised anywhere else in
the application, only in a celery task. Came across this situation trying
to understand why I was not getting error emails for celery tasks.
Triggered this exception by adding :
CELERY_WORKER_HIJACK_ROOT_LOGGER = False
Implying otherwise the error is not redirected to the AdminEmailHandler.
Django version : 1.10.2
Celery version : 4.0.0
(stacktrace is below)
The method "get_traceback_frame_variables" is the cause of this problem.
By wrapping a try except around the code of the method and returning an
empty array, I roughly get what I expect : an email describing the error
with the adequate traceback.
My logging setup :
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'task_id': {
'()': 'lib.logging.task_filter.TaskIDFilter'
},
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters' : {
'task': {
'format': LOGGING_TASK_FORMAT
},
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler',
'include_html': True,
'filters': ['require_debug_false'], # commented or not, the
error happens with DEBUG = True or False
},
},
'loggers': {
'': {
'handlers': ['console', 'mail_admins'],
'level': 'INFO',
'propagate': True
},
}
}
Here is the stacktrace :
Traceback (most recent call last):
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/celery/app/trace.py", line 382, in trace_task
I, R, state, retval = on_error(task_request, exc, uuid)
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/celery/app/trace.py", line 324, in on_error
task, request, eager=eager, call_errbacks=call_errbacks,
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/celery/app/trace.py", line 158, in handle_error_state
call_errbacks=call_errbacks)
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/celery/app/trace.py", line 212, in handle_failure
self._log_error(task, req, einfo)
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/celery/app/trace.py", line 242, in _log_error
extra={'data': context})
File "/usr/lib/python3.4/logging/__init__.py", line 1341, in log
self._log(level, msg, args, **kwargs)
File "/usr/lib/python3.4/logging/__init__.py", line 1409, in _log
self.handle(record)
File "/usr/lib/python3.4/logging/__init__.py", line 1419, in handle
self.callHandlers(record)
File "/usr/lib/python3.4/logging/__init__.py", line 1481, in
callHandlers
hdlr.handle(record)
File "/usr/lib/python3.4/logging/__init__.py", line 853, in handle
self.emit(record)
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/django/utils/log.py", line 119, in emit
message = "%s\n\n%s" % (self.format(no_exc_record),
reporter.get_traceback_text())
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/django/views/debug.py", line 325, in get_traceback_text
c = Context(self.get_traceback_data(), autoescape=False,
use_l10n=False)
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/django/views/debug.py", line 264, in get_traceback_data
frames = self.get_traceback_frames()
File "/home/simon/code/caravan/fibre/env-aws/lib/python3.4/site-
packages/django/views/debug.py", line 420, in get_traceback_frames
'vars': self.filter.get_traceback_frame_variables(self.request,
tb.tb_frame),
File "/home/simon/code/caravan
