Thanks Adam,I'm running the 2.5 beta. Given that I've not yet fully scanned my file system, I think I'm going to wipe the database, reconfigure MySQL as you suggested (also running the perl script)... That way I'll start fresh after the changes.
I'll reply back once I start scanning again to report the entries to namespace size comparison.
Mike ------------------------------------------------------------------------ *From: * Adam Brenner <[email protected]> *Sent: * Tue, 10 Dec 2013 00:36:41 -0800 *To: * Mike Hanby <[email protected]> *Subject: * Re: [robinhood-support] MySQL ibdata1 Size
Mike, This is a normal size. Its about the total number of entries and not the size of the filesystem. On our cluster, we have roughly 50 million entries for a few filesystems we have setup on our cluster: Gluster, FhgFS and some NFS mounts. With MySQL, and InnoDB, the default namespace for ibdata1 is in a shared namespace for each database. The shared namespace can grow in size, but never decrease in size. Because of this, its recommend you enable the following option within your MySQL setup: innodb_file_per_table=1 This will create a separate ibdata file for each database. While it does not solve the shrinking problem, if you delete the database, you can reclaim the space. Also, if you have added custom indexes to your table, this will also increase the size. By the way, what version of RBH are you running? Thomas just posted the 2.5 beta. It would be interesting to see if the file size decreases -- I believe it will. I will be trying out the beta in the upcoming weeks. In addition to setting innodb_file_per_table, might be worth while to mysqltuner.pl on your MySQL server. It will help identify areas where items should be tuned for. Let us know how it goes, -Adam -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ [email protected] On Mon, Dec 9, 2013 at 10:09 AM, Mike Hanby <[email protected]> wrote:Howdy, Are there any suggestions for how large a file system we need to host the Robinhood database for, say, a 100TB Lustre 1.8 file system? I initially set aside 30GB, started my initial scan, and by the time Robinhood had scanned 32 million entries, the file system filled up. The primary usage is the /var/lib/mysql/ibdata1 which is 28GB in size. Currently, the MySQL database is only hosting Robinhood scanning a single Lustre file system, so all of the data in ibdata1 belongs to it. Is this normal for that many entries? Or do I have some serious MySQL tuning to do? Thanks, Mike ------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk _______________________________________________ robinhood-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/robinhood-support
------------------------------------------------------------------------------ Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________ robinhood-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/robinhood-support
