Brock, What version of RBH are you running? What is your commit_behavior set to? I believe if you go ahead and add the following line to your RBH configuration file within the ListManager and then within the MySQL section, this should get you covered:
innodb = disabled; You will then need to convert your database structure to MyISAM. It might be easier to just to delete the database and start from fresh. RBH should create the DB structure with MyISAM engine. Someone can correct me on this if I am wrong. However, what I am guessing is happening (and this is a big guess) is that MySQL is not tuned for InnoDB. When I first setup RBH on our HPC cluster, I noticed the same issue. Writes are extremely slow - double digits insert records being reported by RBH. Here are a few things I did to increase that to three/four digits record inserts (and hopefully this information will help others). Run MySQL Tuner[1]. Make sure this is ran after an initial full scan. This will help you spot areas within the my.cnf configuration file to update. If your system has plenty of memory, take advantage of that. Setting innodb_file_per_table=1 would be a good idea, since each table will get their own namespace for InnoDB rather then sharing the default with every other database. Ideally, you want to keep the entire Innodb index in memory, innodb_buffer_pool_size will do that. Another thing you should do is read the guide Thomas posted earlier about optimizing your RBH workflow[2]. Ideally, you want your bottle neck to be the insert operation (STAGE_DB_APPLY) and not anything else. You will need to fine tune the parameters to make this happen. You might even want to consider lowering (or increasing) max_pending_operations within EntryProcess. Hope that helps, -Adam [1]: http://mysqltuner.pl [2]: http://sourceforge.net/apps/trac/robinhood/wiki/TunePipeline -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ [email protected] On Tue, Dec 3, 2013 at 7:52 AM, Brock Palen <[email protected]> wrote: > Is there a way in newer versions of robinhood to go back to the old MyISAM > tables and settings used in pervious versions? > > We, as many sites do, still run lustre 1.8 and thus must to full scans, in > pervious versions of robinhood on the same hardware we had 1500 updates/s > now we are in a few hundred, the database is so slammed (note we run on cheap > spindles) it makes robinhood useless, scans take multiple days and because > the disk is so busy we can't run any meaningful queries. > > Any thoughts on this would be helpful, I have been on the list before and we > have messed with the mysql settings a lot to try to speed up the scan to no > success. > > Brock Palen > www.umich.edu/~brockp > CAEN Advanced Computing > XSEDE Campus Champion > [email protected] > (734)936-1985 > > > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk > _______________________________________________ > robinhood-support mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/robinhood-support > ------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk _______________________________________________ robinhood-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/robinhood-support
