Thomas, I tried the new build, while it was slightly faster (200/sec rather than 50/sec) and the errors from the current 2.5.0 beta were resolved, I enabled innodb=disabled,
And I am getting still lower performance than 2.3 by about 2-3x but it is still 2000-3000/second and this will work for us. Brock Palen www.umich.edu/~brockp CAEN Advanced Computing XSEDE Campus Champion [email protected] (734)936-1985 On Jan 9, 2014, at 10:31 AM, LEIBOVICI Thomas <[email protected]> wrote: > Hi Brock, > > On 01/09/14 15:49, Brock Palen wrote: >> Adam, >> >> I tried adding innodb=disabled; >> >> To ListManager but 2.5.0 beta barfs on it: >> >> Config Check | WARNING: unknown parameter 'innodb' in block 'ListManager' >> line 74 > This parameter must be specified in the MySQL subblock (not directly in > ListManager), as is it MySQL related: > > ListManager { > ... > MySQL { > ... > innodb = disabled ; > } > } > >> I sadly found out that due to a mistake on my part 2.5.0 was built without >> lustre support so no stripe info was being collected, >> >> When I rebuilt performance was only 50entires/ms :-( Scans never >> finish, you can't run qurries. >> >> My solution for now is to roll all the way back to 2.3.x The last version >> of robinhood that worked at full speed for us. We will look at thi sall >> again when we have lustre + changelogs. It is sad because we really wanted >> the post scan script hook, we are working on a system where we use squop to >> pull all the data from robinhood into hive/pig and then run a bunch of stats >> on our system over time. Problem is robinhood isn't fast enough on the >> hardware I have available (7200RPM sata). > I suggest you give a try to the latest version of code from the git > repository. > If you run it, drop any previous tuning in the EntryProcessor block. You can > also re-enable acct_* parameters. > > With the current code, we get a nice scan speed (6.7M entries per hour > ~1800/sec) and no longer state deadlock issues. > It reached a changelog processing speed of 36k/sec without any sign of > saturation. > The DB is also on a local spinning disk and we use the innodb engine. > > Here are the tunings in /etc/my.cnf. > We recently found that "innodb_log_file_size" has a major impact (see URL > below). > > key_buffer_size=512M > thread_cache_size=64 > query_cache_size=512M > query_cache_limit=512M > sort_buffer_size=512M > read_rnd_buffer_size=1M > table_cache=8K > tmp_table_size=1G > max_heap_table_size=1G > > innodb_file_per_table > innodb_buffer_pool_size=128G# up to 80% of physical memory > innodb_max_dirty_pages_pct=20 > > # see the following tutorial to tune innodb_log_file_size: > #http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size > innodb_log_file_size=900M > > Regards > Thomas > >
signature.asc
Description: Message signed with OpenPGP using GPGMail
------------------------------------------------------------------------------ CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments & Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________ robinhood-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/robinhood-support
