Re: [Django] #12007: djangoadmin loaddata gives odd error if it runs out of memory

[Django]

#12007: djangoadmin loaddata gives odd error if it runs out of memory
-+-
   Reporter:  chrisw |Owner:  nobody
   Type: |   Status:  new
  Cleanup/optimization   |Component:  Core (Serialization)
  Milestone: | Severity:  Normal
Version:  1.1| Keywords:
 Resolution: |Has patch:  0
   Triage Stage:  Accepted   |  Needs tests:  0
Needs documentation:  0  |
Patch needs improvement:  0  |
-+-
Changes (by baumer1122):

 * type:   => Cleanup/optimization
 * severity:   => Normal


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #12007: djangoadmin loaddata gives odd error if it runs out of memory

[Django]

#12007: djangoadmin loaddata gives odd error if it runs out of memory
+---
  Reporter:  chrisw | Owner:  nobody
Status:  new| Milestone:
 Component:  Serialization  |   Version:  1.1   
Resolution: |  Keywords:
 Stage:  Accepted   | Has_patch:  0 
Needs_docs:  0  |   Needs_tests:  0 
Needs_better_patch:  0  |  
+---
Old description:

> Hi All,
>
> I tried to import a 3.2GB xml file, partly to see if Django loads the
> whole file into memory when it's importing and partly to see how large pg
> client transactions work. Whichever caused the problem, I have a fair
> idea that even thought the machine has 4GB of memory, the process ran out
> of memory.
>
> I'd expect a MemoryError if that were the case, what I actually got was:
>
>   File "django/core/management/__init__.py", line 362, in execute_manager
> utility.execute()
>   File "/django/core/management/__init__.py", line 303, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
>   File "django/core/management/base.py", line 195, in run_from_argv
> self.execute(*args, **options.__dict__)
>   File "django/core/management/base.py", line 222, in execute
> output = self.handle(*args, **options)
>   File "django/core/management/commands/loaddata.py", line 200, in handle
> transaction.leave_transaction_management()
>   File "django/db/transaction.py", line 74, in
> leave_transaction_management
> raise TransactionManagementError("This code isn't under transaction
> management")
> django.db.transaction.TransactionManagementError: This code isn't under
> transaction management
>
> cheers,
>
> Chris

New description:

 Hi All,

 I tried to import a 3.2GB xml file, partly to see if Django loads the
 whole file into memory when it's importing and partly to see how large pg
 client transactions work. Whichever caused the problem, I have a fair idea
 that even thought the machine has 4GB of memory, the process ran out of
 memory.

 I'd expect a MemoryError if that were the case, what I actually got was:
 {{{
   File "django/core/management/__init__.py", line 362, in execute_manager
 utility.execute()
   File "/django/core/management/__init__.py", line 303, in execute
 self.fetch_command(subcommand).run_from_argv(self.argv)
   File "django/core/management/base.py", line 195, in run_from_argv
 self.execute(*args, **options.__dict__)
   File "django/core/management/base.py", line 222, in execute
 output = self.handle(*args, **options)
   File "django/core/management/commands/loaddata.py", line 200, in handle
 transaction.leave_transaction_management()
   File "django/db/transaction.py", line 74, in
 leave_transaction_management
 raise TransactionManagementError("This code isn't under transaction
 management")
 django.db.transaction.TransactionManagementError: This code isn't under
 transaction management
 }}}
 cheers,

 Chris

Comment (by Alex):

 Please use preview

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

Re: [Django] #12007: djangoadmin loaddata gives odd error if it runs out of memory

[Django]

#12007: djangoadmin loaddata gives odd error if it runs out of memory
+---
  Reporter:  chrisw | Owner:  nobody
Status:  new| Milestone:
 Component:  Serialization  |   Version:  1.1   
Resolution: |  Keywords:
 Stage:  Accepted   | Has_patch:  0 
Needs_docs:  0  |   Needs_tests:  0 
Needs_better_patch:  0  |  
+---
Changes (by russellm):

  * needs_better_patch:  => 0
  * stage:  Unreviewed => Accepted
  * component:  Uncategorized => Serialization
  * needs_tests:  => 0
  * needs_docs:  => 0

Comment:

 For those looking to dig into this, it's important to identify whether
 this is a database issue (i.e., the database runs out of memory if you try
 to do a 3GB transaction) or a Django issue (i.e., a problem with our
 wrapper loading the data file into memory).

 Its also important to note the analog with #5423 - that ticket deals with
 making dumpdata behave as an iterator. I'm not marking this as a dupe
 because the solution won't necessarily be the same, but it's worth noting
 if someone wants to tackle both problems.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---

[Django] #12007: djangoadmin loaddata gives odd error if it runs out of memory

[Django]

#12007: djangoadmin loaddata gives odd error if it runs out of memory
---+
 Reporter:  chrisw |   Owner:  nobody
   Status:  new|   Milestone:
Component:  Uncategorized  | Version:  1.1   
 Keywords: |   Stage:  Unreviewed
Has_patch:  0  |  
---+
 Hi All,

 I tried to import a 3.2GB xml file, partly to see if Django loads the
 whole file into memory when it's importing and partly to see how large pg
 client transactions work. Whichever caused the problem, I have a fair idea
 that even thought the machine has 4GB of memory, the process ran out of
 memory.

 I'd expect a MemoryError if that were the case, what I actually got was:

   File "django/core/management/__init__.py", line 362, in execute_manager
 utility.execute()
   File "/django/core/management/__init__.py", line 303, in execute
 self.fetch_command(subcommand).run_from_argv(self.argv)
   File "django/core/management/base.py", line 195, in run_from_argv
 self.execute(*args, **options.__dict__)
   File "django/core/management/base.py", line 222, in execute
 output = self.handle(*args, **options)
   File "django/core/management/commands/loaddata.py", line 200, in handle
 transaction.leave_transaction_management()
   File "django/db/transaction.py", line 74, in
 leave_transaction_management
 raise TransactionManagementError("This code isn't under transaction
 management")
 django.db.transaction.TransactionManagementError: This code isn't under
 transaction management

 cheers,

 Chris

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en
-~--~~~~--~~--~--~---