On Mon, Mar 3, 2008 at 8:56 PM, Michael Bayer <[EMAIL PROTECTED]> wrote: > > > On Mar 3, 2008, at 9:36 PM, Lukasz Szybalski wrote: > > > I'll add > > sqlalchemy.pool_recycle = 3600 to my prod.cfg (turbogears) > > > > How is the pool recycle going to affect performance? > > used properly, pool recycle has a negligible effect on performance. > every X amount of seconds (which you should configure to be like, an > hour), a connection in the pool will be closed and a new one reopened > on the next grab. > > > > It seems to me > > that if 1000 people hit my website it will open 1000 connection to > > mysql. > > only if your webserver could handle 1000 simultaneous threads (it cant > unless you have a hundred gigs of ram)....but even in that case the > pool throttles at 15 connections by default so simutaneous requests > over that number would have to wait to get a DB connection. You'd > want to set pool_size and max_overflow to adjust this. > > > > Assuming limiter number of connections my website will shut > > down. How does that change with pool_recycle? > > pool_recycle has nothing to do with how many simultaneous requests can > be handled, it just controls the lifecycle of connections that are > pooled. > > > > > What happens sqlalchemy to mysql connections if if I set: > > server.thread_pool = 30 > > assuming that determines how many simultaneous threads handle > requests, the number of connections opened would most likely max at 30 > if you set the pool_size/max_overflow to allow it, and also you only > used one connection per request (i.e. didnt open a second one to write > to a different transaction or something). >
Actually I found that mysql was closed due to system "out of memory" So I guess my question is. What do I need to control so my system so it doesn't run out of memory? Following is from a top command. 27691 www-data 15 0 56680 29m 6960 S 0.0 5.8 0:07.19 apache2 27887 www-data 18 0 56680 29m 6960 S 0.0 5.8 0:00.00 apache2 27692 www-data 16 0 57868 30m 6736 S 0.0 6.1 0:05.08 apache2 27820 www-data 18 0 57868 30m 6736 S 0.0 6.1 0:00.00 apache2 27694 www-data 15 0 76048 40m 7108 S 0.0 8.0 0:15.01 apache2 27914 www-data 18 0 76048 40m 7108 S 0.0 8.0 0:00.00 apache2 27916 www-data 21 0 76048 40m 7108 S 0.0 8.0 0:00.00 apache2 27695 www-data 17 0 58404 31m 7148 S 0.0 6.2 0:13.49 apache2 27910 www-data 17 0 58404 31m 7148 S 0.0 6.2 0:00.00 apache2 27698 www-data 15 0 76532 41m 7048 S 0.0 8.1 0:15.00 apache2 27818 www-data 18 0 76532 41m 7048 S 0.0 8.1 0:00.00 apache2 27904 www-data 21 0 76532 41m 7048 S 0.0 8.1 0:00.00 apache2 27817 www-data 15 0 73316 36m 5276 S 0.0 7.2 0:09.54 apache2 27906 www-data 17 0 73316 36m 5276 S 0.0 7.2 0:00.00 apache2 27918 www-data 21 0 73316 36m 5276 S 0.0 7.2 0:00.00 apache2 27871 www-data 15 0 55680 27m 5184 S 0.0 5.4 0:09.50 apache2 27912 www-data 18 0 55680 27m 5184 S 0.0 5.4 0:00.00 apache2 27957 www-data 15 0 70532 33m 5168 S 0.0 6.7 0:04.42 apache2 27962 www-data 21 0 70532 33m 5168 S 0.0 6.7 0:00.00 apache2 28028 www-data 18 0 70532 33m 5168 S 0.0 6.7 0:00.00 apache2 27958 www-data 15 0 56516 27m 5308 S 0.0 5.5 0:06.26 apache2 27976 www-data 18 0 56516 27m 5308 S 0.0 5.5 0:00.00 apache2 27959 www-data 17 0 55376 25m 5180 S 0.0 5.1 0:02.47 apache2 27963 www-data 21 0 55376 25m 5180 S 0.0 5.1 0:00.00 apache2 At some point the system runs out of memory and starts killing programs aka mysql. Mar 4 15:56:17 myserver kernel: Out of Memory: Kill process 27421 (mysqld) score 32665 and children. Mar 4 15:56:17 myserver kernel: Out of memory: Killed process 27421 (mysqld). Mar 4 15:56:18 myserver mysqld_safe[27488]: Number of processes running now: 0 Mar 4 15:56:18 myserver mysqld_safe[27490]: restarted Mar 4 15:56:19 myserver mysqld[27493]: 080304 15:56:19 InnoDB: Started; log sequence number 0 43655 Mar 4 15:56:19 myserver mysqld[27493]: 080304 15:56:19 [Note] Recovering after a crash using /var/log/mysql/mysql-bin Mar 4 15:56:19 myserver mysqld[27493]: 080304 15:56:19 [Note] Starting crash recovery... Mar 4 15:56:19 myserver mysqld[27493]: 080304 15:56:19 [Note] Crash recovery finished. Mar 4 15:56:19 myserver mysqld[27493]: 080304 15:56:19 [Note] /usr/sbin/mysqld: ready for connections. Mar 4 15:56:19 myserver mysqld[27493]: Version: '5.0.32-Debian_7etch1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Debian etch distribution Mar 4 17:40:41 myserver kernel: Out of Memory: Kill process 27492 (mysqld) score 31974 and children. Mar 4 17:40:41 myserver kernel: Out of memory: Killed process 27492 (mysqld). Mar 4 17:40:41 myserver kernel: Out of Memory: Kill process 27499 (mysqld) score 31974 and children. Mar 4 17:40:41 myserver kernel: Out of memory: Killed process 27499 (mysqld). Mar 4 17:40:41 myserver kernel: Out of Memory: Kill process 27500 (mysqld) score 31974 and children. Mar 4 17:40:41 myserver kernel: Out of memory: Killed process 27500 (mysqld). Mar 4 17:40:43 myserver mysqld_safe[27608]: Number of processes running now: 0 Mar 4 17:40:43 myserver mysqld_safe[27610]: restarted Mar 4 17:40:47 myserver mysqld[27613]: InnoDB: Error: pthread_create returned 12 Mar 4 17:40:48 myserver mysqld_safe[27620]: ended Mar 4 18:13:05 myserver mysqld_safe[27764]: started What can mod_wsgi do to help? What can turbogears do to help? What can sqlalchemy do to help? Thanks, Lucas --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
