Re: Django error page - MemoryError
Anyone else have any thoughts on if I should submit this for consideration into the core? Cal On Tue, Dec 20, 2011 at 8:16 PM, Cal Leeming [Simplicity Media Ltd] < cal.leem...@simplicitymedialtd.co.uk> wrote: > Comments below, apologies for the email signature on previous emails! > > On Tue, Dec 20, 2011 at 7:56 PM, Paul McMillan wrote: > >> > Place a try/catch for MemoryError on the exception handler to send back >> a >> > simple exception traceback to the browser. >> >> Yes, this makes sense, as long as we are sure the memory error is >> raised by Django code, not user code. >> > > Agreed, this would be wrapped around get_traceback_html() most likely. > > >> >> > Include a configuration settings option to limit the maximum payload it >> will >> > send back to the browser per variable (i.e. maybe 500kb per stack >> frame, or >> > 2kb per variable etc) >> >> I think we should select some reasonable limits for these, and >> hardcode them, rather than adding a setting. Users who are debugging >> the entire contents of multi-megabyte variable values on the html >> debug page are doing it wrong. >> > > I completely agree. > > I'm wondering if it would be better to have the variable commented out > with a small message if it's above X amount, or whether it should display > up to X amount, then display a small message explaining why it's been > clipped. > > >> >> -Paul >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To post to this group, send email to django-developers@googlegroups.com. >> To unsubscribe from this group, send email to >> django-developers+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-developers?hl=en. >> >> > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django error page - MemoryError
Comments below, apologies for the email signature on previous emails! On Tue, Dec 20, 2011 at 7:56 PM, Paul McMillan wrote: > > Place a try/catch for MemoryError on the exception handler to send back a > > simple exception traceback to the browser. > > Yes, this makes sense, as long as we are sure the memory error is > raised by Django code, not user code. > Agreed, this would be wrapped around get_traceback_html() most likely. > > > Include a configuration settings option to limit the maximum payload it > will > > send back to the browser per variable (i.e. maybe 500kb per stack frame, > or > > 2kb per variable etc) > > I think we should select some reasonable limits for these, and > hardcode them, rather than adding a setting. Users who are debugging > the entire contents of multi-megabyte variable values on the html > debug page are doing it wrong. > I completely agree. I'm wondering if it would be better to have the variable commented out with a small message if it's above X amount, or whether it should display up to X amount, then display a small message explaining why it's been clipped. > > -Paul > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django error page - MemoryError
Hi Alex, Please note, I am already using a try/catch block on MemoryError, and this does indeed resolve the problem. I think at the very least, we should attempt to generate the text exception, and if it fails due to a particular circumstance, then it will just fall back to doing whatever it originally does. I agree that it won't always 100% work (although I'm yet to see it fail in my tests), but generating the text based exception has a fairly small footprint, so it should work the majority of the time. Cal On Tue, Dec 20, 2011 at 8:01 PM, Alex Gaynor wrote: > > > On Tue, Dec 20, 2011 at 1:56 PM, Paul McMillan wrote: > >> > Place a try/catch for MemoryError on the exception handler to send back >> a >> > simple exception traceback to the browser. >> >> Yes, this makes sense, as long as we are sure the memory error is >> raised by Django code, not user code. >> >> > Include a configuration settings option to limit the maximum payload it >> will >> > send back to the browser per variable (i.e. maybe 500kb per stack >> frame, or >> > 2kb per variable etc) >> >> I think we should select some reasonable limits for these, and >> hardcode them, rather than adding a setting. Users who are debugging >> the entire contents of multi-megabyte variable values on the html >> debug page are doing it wrong. >> >> -Paul >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers" group. >> To post to this group, send email to django-developers@googlegroups.com. >> To unsubscribe from this group, send email to >> django-developers+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-developers?hl=en. >> >> > Once you get a real memory error (i.e. one indicating you're out of > memory, not just some operation wouldn't work "" * sys.maxint for > example), doing *any* allocation might fail, so any limits you try to put > down won't work in all circumstances. > > Alex > > -- > "I disapprove of what you say, but I will defend to the death your right > to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > -- -- *Cal Leeming* Technical Director | www.simplicitymedialtd.co.uk Registered company number 7143564 Office: +44 (0)2476 980798 | Cell: +44 (0)7534 971120 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django error page - MemoryError
On Tue, Dec 20, 2011 at 1:56 PM, Paul McMillan wrote: > > Place a try/catch for MemoryError on the exception handler to send back a > > simple exception traceback to the browser. > > Yes, this makes sense, as long as we are sure the memory error is > raised by Django code, not user code. > > > Include a configuration settings option to limit the maximum payload it > will > > send back to the browser per variable (i.e. maybe 500kb per stack frame, > or > > 2kb per variable etc) > > I think we should select some reasonable limits for these, and > hardcode them, rather than adding a setting. Users who are debugging > the entire contents of multi-megabyte variable values on the html > debug page are doing it wrong. > > -Paul > > -- > You received this message because you are subscribed to the Google Groups > "Django developers" group. > To post to this group, send email to django-developers@googlegroups.com. > To unsubscribe from this group, send email to > django-developers+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-developers?hl=en. > > Once you get a real memory error (i.e. one indicating you're out of memory, not just some operation wouldn't work "" * sys.maxint for example), doing *any* allocation might fail, so any limits you try to put down won't work in all circumstances. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Django error page - MemoryError
> Place a try/catch for MemoryError on the exception handler to send back a > simple exception traceback to the browser. Yes, this makes sense, as long as we are sure the memory error is raised by Django code, not user code. > Include a configuration settings option to limit the maximum payload it will > send back to the browser per variable (i.e. maybe 500kb per stack frame, or > 2kb per variable etc) I think we should select some reasonable limits for these, and hardcode them, rather than adding a setting. Users who are debugging the entire contents of multi-megabyte variable values on the html debug page are doing it wrong. -Paul -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Django error page - MemoryError
Hey, So, we have a few clients who use Django for processing large amounts of data in a single query. If an exception is raised in development, the get_traceback_html() method fails with a MemoryError, and in the event that it doesn't, you end up with huge variable data print outs making the debug page somewhat tedious to use. Temporarily, I've put a try/catch on MemoryError to generate a text based email, which at least tells us what originally happened, albeit with the useful variable information. With all this in mind, I would like to see if anyone else thinks the following might be a good idea - Place a try/catch for MemoryError on the exception handler to send back a simple exception traceback to the browser. - Include a configuration settings option to limit the maximum payload it will send back to the browser per variable (i.e. maybe 500kb per stack frame, or 2kb per variable etc) I'd be happy to provide a patch for this, assuming it has any chance of getting into the core. Any thoughts/feedback would be appreciated. Cal -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
