** Description changed: With large /etc/group and /etc/passwd files, duplicity spends the vast majority of its time mapping group and user names to gids and uids. This is with duplicity 0.6.18-0ubuntu3 on Ubuntu 12.04 and Python 2.7.3. I have a largish group map (7MB, with many group entries having lines in the 100KB - 200KB range.) A zero change incremental backup of ~200k files takes about 23 minutes. Profiling the duplicity run shows that 92% of the time is spent calling grp.getgrnam! Another ~2% is spent calling grp.getgrgid, pwd.getpwnam, and pwd.getpwuid. Some simple caching of caching of this information would make a big improvement. + + -- + + SRU Justification: + [Impact] + * This fix should be backported into Precise LTS because users of duplicity with large group/passwd files will encounter performance issues. + + [Test Case] + * Using a large group/passwd file configuration, perform a backup and record the time it takes to backup. Profile this as well and determine how much time is spent in grp.getgrnam. Compare this with the patched version and ensure improvement in performance. + + [Regression Potential] + * This is a backported patch, and is currently in the latest duplicity. However; because it is a new feature we will need to ensure backups work as expected with this patch.
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1013446 Title: Uncached grp and pwd calls make duplicity slow with large group and passwd maps To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1013446/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
