Hi,

Has something significant changed on repo sites?  For years, I have been 
using lftp to selectively mirror parts of these repos to use on Internet 
isolated networks.  lftp is a great tool and a little easier and more 
efficient than wget.  Note, we are not able to use rsync, it is not 
permitted as an outbound protocol.

So, what's happening when I use lftp today?  Every directory on the repo 
looks like a file, not a directory, see below:
$ lftp http://apt.puppetlabs.com
cd ok, cwd=/                                           
lftp apt.puppetlabs.com:/> ls
-rw-r--r--  --  index_by_name_reverse.html
-rw-r--r--  --  index_by_lastModified.html
-rw-r--r--  --  index_by_size.html
-rw-r--r--  --  index_by_name.html
-rw-r--r--  --  DEB-GPG-KEY-puppet
-rw-r--r--  --  DEB-GPG-KEY-puppetlabs
-rw-r--r--  --  DEB-GPG-KEY-reductive
-rw-r--r--  --  README.txt
-rw-r--r--  --  dists
-rw-r--r--  --  keyring.gpg
-rw-r--r--  --  pool
-rw-r--r--  --  pubkey.gpg
...<snip>...

What should happen?
$ lftp http://mirror.centos.org
cd ok, cwd=/                                          
lftp mirror.centos.org:/> ls
drwxr-xr-x  --  /          
-rw-r--r--  --  HEADER.images/modern-CentOS-logo.png
-rw-r--r--  --  icons/blank.gif
-rw-r--r--  --  icons/unknown.gif
-rw-r--r--           11  2018-03-01 16:12  TIME
-rw-r--r--  --  icons/folder.gif
drwxr-xr-x            -  2017-09-06 06:57  altarch
drwxr-xr-x            -  2009-09-09 05:18  centos-2
drwxr-xr-x            -  2007-10-22 17:16  centos-3
drwxr-xr-x            -  2011-03-02 14:00  centos-4
drwxr-xr-x            -  2014-09-29 08:59  centos-5
drwxr-xr-x            -  2017-04-05 11:36  centos-6
drwxr-xr-x            -  2017-09-13 20:25  centos-7
drwxr-xr-x            -  2017-09-13 13:56  centos
-rw-r--r--          132  2018-03-01 16:10  dir_sizes_altarch
-rw-r--r--          924  2018-03-01 16:10  dir_sizes_centos
drwxr-xr-x            -  2014-07-07 12:49  mirrorscripts
-rw-r--r--  --  icons/text.gif
...<snip>...

As a result, my entire mirror of both apt/yum.puppetlabs.com turned into a 
bunch of static files, deleting a heap of data I'm going to have figure out 
how to download again using some other means.  I attempted to reconfigure 
my mirror script to use wget instead of lftp, but it only downloaded html 
versions of the directory indexes, no RPMs, no repodata.

Any ideas or anything as to what's up?

Thanks for the assistance!

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-users/7355d1ea-9a01-4338-ad53-7fa9fc8cc80d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to