Anyone know for sure if the memory needed to insert a LOB is a percentage of the system's available memory or if it is allocated from the innodb_buffer_pool_size? IOW, how should my configuration settings be modified to allow the insertion of larger blobs? :)
-----Original Message----- From: Robert DiFalco [mailto:[EMAIL PROTECTED] Sent: Saturday, April 29, 2006 9:49 AM To: Sergei Golubchik Cc: [EMAIL PROTECTED]; mysql@lists.mysql.com Subject: RE: Streaming LOB Data Thanks Sergei, it's nice to know for sure. Do you know if there is any documentation on how memory is used to store LOB data? For example, is it a percentage of the total buffer pool size or is it allocated from available memory un-allocated to the buffer pool size? -----Original Message----- From: Sergei Golubchik [mailto:[EMAIL PROTECTED] Sent: Saturday, April 29, 2006 9:34 AM To: Robert DiFalco Cc: [EMAIL PROTECTED]; mysql@lists.mysql.com Subject: Re: Streaming LOB Data Hi! On Apr 28, Robert DiFalco wrote: > It appears (for InnoDB at least) that while INSERTing a LOB that all > LOB data must be loaded into memory before it is written to disk. Or > is it just the size of the combined log files? Looking at Task > Manager, it looks like it DOES try to load the whole thing into memory > before streaming it to disk; which just seems wacky. > > The error I get is this: > > "Out of memory (Needed xxx ..." > > If the server crashes I get this: > > InnoDB: which exceeds the log group capacity 18870682. > InnoDB: If you are using big BLOB or TEXT rows, you must set the > InnoDB: combined size of log files at least 10 times bigger than the > InnoDB: largest such row. > 060427 15:26:53 InnoDB: Error: cannot allocate 539001144 bytes of > InnoDB: memory with malloc! Total allocated memory > InnoDB: by InnoDB 52823786 bytes. Operating system errno: 8 > InnoDB: Check if you should increase the swap file or > InnoDB: ulimits of your operating system. > InnoDB: On FreeBSD check you have compiled the OS with > InnoDB: a big enough maximum process size. > InnoDB: Note that in most 32-bit computers the process > InnoDB: memory space is limited to 2 GB or 4 GB. > > How can I make the size of LOBs I insert NOT be memory constrained? No way. Unfortunately, it's the way MySQL works at the moment. Which does not mean it's not going to be changed in the future. Regards, Sergei -- __ ___ ___ ____ __ / |/ /_ __/ __/ __ \/ / Sergei Golubchik <[EMAIL PROTECTED]> / /|_/ / // /\ \/ /_/ / /__ MySQL AB, Senior Software Developer /_/ /_/\_, /___/\___\_\___/ Kerpen, Germany <___/ www.mysql.com -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
