Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On Thu, 2015-10-29 at 08:48 -0600, Ken Dreyer wrote:
> On Wed, Oct 28, 2015 at 7:54 PM, Matt Taylor 
> wrote:
> > I still see rsync errors due to permissions on the remote side:
> > 
> 
> Thanks for the heads' up; I bet another upload rsync process got
> interrupted there.
> 
> I've run the following to remove all the oddly-named RPM files:
> 
>   for f in $(locate *.rpm.* ) ; do rm -i $f; done
> 
> Please let us know if there are other problems like this.

Since a month the rsync is broken again.

———
rsync: send_files failed to open "/debian-jewel/db/lockfile" (in ceph):
Permission denied (13)
rsync: send_files failed to open "/debian-jewel/pool/main/c/ceph/.ceph-
fuse-dbg_10.1.0-1~bpo80+1_amd64.deb.h0JvHM" (in ceph): Permission
denied (13)
rsync: send_files failed to open "/debian-jewel/pool/main/c/ceph/.ceph-
test-dbg_10.1.0-1trusty_amd64.deb.06D0AZ" (in ceph): Permission denied
(13)
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1655) [generator=3.1.1]
———

you fixed this problem last time, so please fix it again :-)

try: 
  find /path/to/mirror -iname '.*.deb.* -ls
if all files that are found are broken:
  find /path/to/mirror -iname '.*.deb.* -delete

Thanks a lot
Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Ken Dreyer]

On Wed, Oct 28, 2015 at 7:54 PM, Matt Taylor  wrote:
> I still see rsync errors due to permissions on the remote side:
>

Thanks for the heads' up; I bet another upload rsync process got
interrupted there.

I've run the following to remove all the oddly-named RPM files:

  for f in $(locate *.rpm.* ) ; do rm -i $f; done

Please let us know if there are other problems like this.

- Ken
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On 10/27/2015 06:21 PM, Ken Dreyer wrote:
> Thanks, I've deleted it from the download.ceph.com web server.

Thanks a lot!
My mirror is up-to-date now.

Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Matt Taylor]

I still see rsync errors due to permissions on the remote side:


rsync: send_files failed to open 
"/rpm-dumpling/rhel6/x86_64/ceph-debuginfo-0.67.8-0.el6.x86_64.rpm.iVHKKi" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-firefly/rhel6/x86_64/ceph-debuginfo-0.80.1-0.el6.x86_64.rpm.aSeb3w" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-firefly/rhel7/x86_64/ceph-debuginfo-0.80.1-0.el7.x86_64.rpm.0CRaPO" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-hammer/el7/x86_64/ceph-debuginfo-0.94-0.el7.centos.x86_64.rpm.gKCcjG" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-hammer/el7/x86_64/ceph-debuginfo-0.94.2-0.el7.centos.x86_64.rpm.SEIHHy" 
(in ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-hammer/el7/x86_64/ceph-debuginfo-0.94.2-0.el7.centos.x86_64.rpm.wxiwwd" 
(in ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-hammer/rhel7/x86_64/ceph-debuginfo-0.94-0.el7.x86_64.rpm.SZn8B0" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-hammer/rhel7/x86_64/ceph-debuginfo-0.94.1-0.el7.x86_64.rpm.UlNVak" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-hammer/rhel7/x86_64/ceph-debuginfo-0.94.1-0.el7.x86_64.rpm.Z3M5Dz" (in 
ceph): Permission denied (13)
rsync: send_files failed to open 
"/rpm-testing/el6/x86_64/ceph-debuginfo-9.0.3-0.el6.x86_64.rpm.jl0MZj" (in 
ceph): Permission denied (13)



These files have been like this for a while (I became aware on 2015-10-22).

It would be nice if these could be removed as they're not required 
(already uploaded), see:


http://download.ceph.com/rpm-hammer/el7/x86_64/

Cheers,
Matt.

On 28/10/2015 22:16, Björn Lässig wrote:

On 10/27/2015 06:21 PM, Ken Dreyer wrote:

Thanks, I've deleted it from the download.ceph.com web server.


Thanks a lot!
My mirror is up-to-date now.

Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On 10/27/2015 10:22 AM, Wido den Hollander wrote:

On 27-10-15 09:51, Björn Lässig wrote:

after having some problems with ipv6 and download.ceph.com, i made a
mirror (debian-hammer only) for my ipv6-only cluster.


I see you are from Germany, you can also sync from eu.ceph.com


good to know, that you give rsync access, too.
The Problem here is that there is no indication, that the primary mirror 
(download.ceph.com) is in a complete state. I suppose you you use some 
sort of cron-method for synchronizing with download.c.c. So in case the 
primary ist broken, the chain needs more time to repair.
Thats why i decided it is more robust to rsync from download.c.c 
directly. i will change this, if eu.c.c is pushed on update by download.c.c



PS: a debian like mirror-push mechanic via
https://ftp-master.debian.org/git/archvsync.git would be very helpful.
This would add traces and Archive-in-progress files and all the other
things, the debian-mirror-admins had to fix over the years. This would
lead to less traffic for you and always up-to-date mirrors for us.


That is still on my TODO list. We need a nice mirror system for Ceph.


i run a secondary debian mirror at my old university for more than 10 
years now. If you start working on that, please keep me in the loop.
Maybe starting with a ceph-mirror mailinglist would be a possibility, to 
concentrate all interested people.


Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Wido den Hollander]

On 27-10-15 09:51, Björn Lässig wrote:
> Hi,
> 
> after having some problems with ipv6 and download.ceph.com, i made a
> mirror (debian-hammer only) for my ipv6-only cluster.
> 

I see you are from Germany, you can also sync from eu.ceph.com

> Unfortunately after the release of 0.94.5 the rsync breaks with:
> 
> # less rsync-ftpsync-mirror_ceph.error.0
> rsync: send_files failed to open
> "/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ"
> (in ceph): Permission denied (13)
> rsync error: some files/attrs were not transferred (see previous errors)
> (code 23) at main.c(1655) [generator=3.1.1]
> 
> indeed there is:
> 
> [09:40:49] ~ > rsync -4 -L
> download.ceph.com::ceph/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
> 
> -rw--- 91,488,256 2015/10/26 19:36:46
> .ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
> 
> i would be thankful, if you could remove this broken file or complete
> the mirror process.
> 
> Thanks in advance
> 
> Björn Lässig
> 
> PS: a debian like mirror-push mechanic via
> https://ftp-master.debian.org/git/archvsync.git would be very helpful.
> This would add traces and Archive-in-progress files and all the other
> things, the debian-mirror-admins had to fix over the years. This would
> lead to less traffic for you and always up-to-date mirrors for us.

That is still on my TODO list. We need a nice mirror system for Ceph.

Wido

> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Ken Dreyer]

On Tue, Oct 27, 2015 at 2:51 AM, Björn Lässig  wrote:
> indeed there is:
>
> [09:40:49] ~ > rsync -4 -L
> download.ceph.com::ceph/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
> -rw--- 91,488,256 2015/10/26 19:36:46
> .ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
>
> i would be thankful, if you could remove this broken file or complete the
> mirror process.
>

Alfredo this looks like a leftover from some rsync upload process. I
think we can just delete this file, right?

- Ken
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Ken Dreyer]

Thanks, I've deleted it from the download.ceph.com web server.

- Ken

On Tue, Oct 27, 2015 at 11:06 AM, Alfredo Deza  wrote:
> Yes that file can (should) be deleted
>
> On Tue, Oct 27, 2015 at 12:49 PM, Ken Dreyer  wrote:
>> On Tue, Oct 27, 2015 at 2:51 AM, Björn Lässig  
>> wrote:
>>> indeed there is:
>>>
>>> [09:40:49] ~ > rsync -4 -L
>>> download.ceph.com::ceph/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
>>> -rw--- 91,488,256 2015/10/26 19:36:46
>>> .ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
>>>
>>> i would be thankful, if you could remove this broken file or complete the
>>> mirror process.
>>>
>>
>> Alfredo this looks like a leftover from some rsync upload process. I
>> think we can just delete this file, right?
>>
>> - Ken
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Alfredo Deza]

Yes that file can (should) be deleted

On Tue, Oct 27, 2015 at 12:49 PM, Ken Dreyer  wrote:
> On Tue, Oct 27, 2015 at 2:51 AM, Björn Lässig  
> wrote:
>> indeed there is:
>>
>> [09:40:49] ~ > rsync -4 -L
>> download.ceph.com::ceph/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
>> -rw--- 91,488,256 2015/10/26 19:36:46
>> .ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
>>
>> i would be thankful, if you could remove this broken file or complete the
>> mirror process.
>>
>
> Alfredo this looks like a leftover from some rsync upload process. I
> think we can just delete this file, right?
>
> - Ken
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On 10/27/2015 02:50 PM, Wido den Hollander wrote:

i run a secondary debian mirror at my old university for more than 10
years now. If you start working on that, please keep me in the loop.
Maybe starting with a ceph-mirror mailinglist would be a possibility, to
concentrate all interested people.


Yes, that would be an idea. I can ask for that mailinglist.

But keep in mind, download.ceph.com doesn't only serve DEB, it also
serves tarballs and RPM packages.


yes. thats why we would need help from some fedora/redhat/centos admins. 
I do not know how their stuff is distributed. i am a debian guy since i 
crushed yast a very long time ago.


I see 3 points:
 * give clusteradmins an easy way to get a OS/ceph-version mirror for
   their architecture (this is my use case)
 * doing a full mirror (eu.ceph.com)
 * inform mirror-owners (push mirrors), if something changed

Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Wido den Hollander]

On 27-10-15 11:45, Björn Lässig wrote:
> On 10/27/2015 10:22 AM, Wido den Hollander wrote:
>> On 27-10-15 09:51, Björn Lässig wrote:
>>> after having some problems with ipv6 and download.ceph.com, i made a
>>> mirror (debian-hammer only) for my ipv6-only cluster.
>>
>> I see you are from Germany, you can also sync from eu.ceph.com
> 
> good to know, that you give rsync access, too.
> The Problem here is that there is no indication, that the primary mirror
> (download.ceph.com) is in a complete state. I suppose you you use some
> sort of cron-method for synchronizing with download.c.c. So in case the
> primary ist broken, the chain needs more time to repair.
> Thats why i decided it is more robust to rsync from download.c.c
> directly. i will change this, if eu.c.c is pushed on update by download.c.c
> 
>>> PS: a debian like mirror-push mechanic via
>>> https://ftp-master.debian.org/git/archvsync.git would be very helpful.
>>> This would add traces and Archive-in-progress files and all the other
>>> things, the debian-mirror-admins had to fix over the years. This would
>>> lead to less traffic for you and always up-to-date mirrors for us.
>>
>> That is still on my TODO list. We need a nice mirror system for Ceph.
> 
> i run a secondary debian mirror at my old university for more than 10
> years now. If you start working on that, please keep me in the loop.
> Maybe starting with a ceph-mirror mailinglist would be a possibility, to
> concentrate all interested people.
> 
Yes, that would be an idea. I can ask for that mailinglist.

But keep in mind, download.ceph.com doesn't only serve DEB, it also
serves tarballs and RPM packages.

Wido

> Björn
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com