Hi, Has something significant changed on repo sites? For years, I have been using lftp to selectively mirror parts of these repos to use on Internet isolated networks. lftp is a great tool and a little easier and more efficient than wget. Note, we are not able to use rsync, it is not permitted as an outbound protocol.
So, what's happening when I use lftp today? Every directory on the repo looks like a file, not a directory, see below: $ lftp http://apt.puppetlabs.com cd ok, cwd=/ lftp apt.puppetlabs.com:/> ls -rw-r--r-- -- index_by_name_reverse.html -rw-r--r-- -- index_by_lastModified.html -rw-r--r-- -- index_by_size.html -rw-r--r-- -- index_by_name.html -rw-r--r-- -- DEB-GPG-KEY-puppet -rw-r--r-- -- DEB-GPG-KEY-puppetlabs -rw-r--r-- -- DEB-GPG-KEY-reductive -rw-r--r-- -- README.txt -rw-r--r-- -- dists -rw-r--r-- -- keyring.gpg -rw-r--r-- -- pool -rw-r--r-- -- pubkey.gpg ...<snip>... What should happen? $ lftp http://mirror.centos.org cd ok, cwd=/ lftp mirror.centos.org:/> ls drwxr-xr-x -- / -rw-r--r-- -- HEADER.images/modern-CentOS-logo.png -rw-r--r-- -- icons/blank.gif -rw-r--r-- -- icons/unknown.gif -rw-r--r-- 11 2018-03-01 16:12 TIME -rw-r--r-- -- icons/folder.gif drwxr-xr-x - 2017-09-06 06:57 altarch drwxr-xr-x - 2009-09-09 05:18 centos-2 drwxr-xr-x - 2007-10-22 17:16 centos-3 drwxr-xr-x - 2011-03-02 14:00 centos-4 drwxr-xr-x - 2014-09-29 08:59 centos-5 drwxr-xr-x - 2017-04-05 11:36 centos-6 drwxr-xr-x - 2017-09-13 20:25 centos-7 drwxr-xr-x - 2017-09-13 13:56 centos -rw-r--r-- 132 2018-03-01 16:10 dir_sizes_altarch -rw-r--r-- 924 2018-03-01 16:10 dir_sizes_centos drwxr-xr-x - 2014-07-07 12:49 mirrorscripts -rw-r--r-- -- icons/text.gif ...<snip>... As a result, my entire mirror of both apt/yum.puppetlabs.com turned into a bunch of static files, deleting a heap of data I'm going to have figure out how to download again using some other means. I attempted to reconfigure my mirror script to use wget instead of lftp, but it only downloaded html versions of the directory indexes, no RPMs, no repodata. Any ideas or anything as to what's up? Thanks for the assistance! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7355d1ea-9a01-4338-ad53-7fa9fc8cc80d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
