Issue #1469 has been updated by masterzen. Status changed from Accepted to Ready for Testing Assigned to changed from masterzen to jamtur01 Priority changed from High to Normal Target version set to 0.25.0 % Done changed from 0 to 100 Complexity changed from Unknown to Easy Affected version changed from 0.25.0 to 0.24.8
Hi, It took 5 months :-) but the patch is now available for master in my github repository (see tickets/1469): http://github.com/masterzen/puppet/tree/tickets/1469 Thanks, ---------------------------------------- Bug #1469: File recursion with a remote source should not recurse locally http://projects.reductivelabs.com/issues/1469 Author: masterzen Status: Ready for Testing Priority: Normal Assigned to: jamtur01 Category: file Target version: 0.25.0 Complexity: Easy Patch: None Affected version: 0.24.8 Keywords: file recursive slow Hi, I have the following manifests: <pre> file { "${homeroot}/$name": source => [ "puppet:///files/users/$name/skel/", "puppet:///files/users/skel/"], sourceselect => all, checksum => mtime, recurse => true, owner => $name, group => $name, replace => no, require => [ User["$name"], Group["$name"] ]; } </pre> that is used to copy various files to the home directory of a new user. If the home directory of this user contains a deep hierarchy full of files, puppet takes ages and consume a lot of memory to finally copy over a handful of files. My understanding of the issue is the following: * Puppet builds localrecurse first (see puppet/type/file.rb). This parts can take a long time, each local present file has to be stat'ed and a new (implicit) resource is created in the catalog. * then Puppet builds the sourcerecurse If Puppet would build the sourcerecurse first, and prune the localrecurse in function of the availability of the sourced file/dir it would be way much faster. Moreover I don't think people are using file{} to distribute more than a few files, but I bet they like me have more local files than distributed. Also note, that there is another issue if filebucket and checksum are enabled in the aforementioned manifest: it will copy over all files, and filebucket the previous one, as seen in this log extract: <pre> debug: Calling fileserver.list debug: Calling fileserver.describe debug: Calling fileserver.retrieve debug: //all_users/Account[brice]/File[/home/brice/.zshrc]/source: Executing 'diff -u /home/brice/.zshrc /tmp/user/0/puppet-diffing20080704-15708-glfegc-0' debug: //all_users/Account[brice]/File[/home/brice/.zshrc]: Changing checksum,source debug: //all_users/Account[brice]/File[/home/brice/.zshrc]: 2 change(s) debug: //all_users/Account[brice]/File[/home/brice/.zshrc]/checksum: Replacing /home/brice/.zshrc checksum {md5}1395a1f669e2c86443b142cc5ad10f55 with {mtime}Fri Jul 04 15:17:39 +0200 2008 notice: //all_users/Account[brice]/File[/home/brice/.zshrc]/checksum: checksum changed '{md5}1395a1f669e2c86443b142cc5ad10f55' to '{mtime}Fri Jul 04 15:17:39 +0200 2008' debug: Calling puppetbucket.addfile info: //all_users/Account[brice]/File[/home/brice/.zshrc]: Filebucketed to server with sum 1395a1f669e2c86443b142cc5ad10f55 debug: //all_users/Account[brice]/File[/home/brice/.zshrc]/checksum: Replacing /home/brice/.zshrc checksum {mtime}Fri Jul 04 15:17:39 +0200 2008 with {md5}1395a1f669e2c86443b142cc5ad10f55 notice: //all_users/Account[brice]/File[/home/brice/.zshrc]/source: replacing from source puppet:///files/users/brice/skel/.zshrc with contents {md5}1395a1f669e2c86443b142cc5ad10f55 </pre> This is because default file checksum is md5, but my file resource above said "mtime". It then compares apples to oranges. ---------------------------------------- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://reductivelabs.com/redmine/my/account --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en -~----------~----~----~----~------~----~------~--~---
