[ceph-users] rsync service download.ceph.com partially broken

[Björn Lässig]

Hi cephers,

 since some time i get errors while rsyncing from the ceph download server:

download.ceph.com:

rsync: send_files failed to open "/debian-jewel/db/lockfile" (in ceph): 
Permission denied (13)
"/debian-jewel/pool/main/c/ceph/.ceph-fuse-dbg_10.1.0-1~bpo80+1_amd64.deb.h0JvHM"
 (in ceph): Permission denied (13)
rsync: send_files failed to open 
"/debian-jewel/pool/main/c/ceph/.ceph-test-dbg_10.1.0-1trusty_amd64.deb.06D0AZ" 
(in ceph): Permission denied (13)

on eu.ceph.com there are some other files broken:

rsync: send_files failed to open 
"/debian-jewel/pool/main/c/ceph/.ceph-test-dbg_10.2.4-1trusty_amd64.deb.BnXWIa" 
(in ceph): Permission denied (13)
rsync: send_files failed to open 
"/debian-jewel/pool/main/c/ceph/.ceph-test-dbg_10.2.4-1xenial_amd64.deb.5Xhv3J" 
(in ceph): Permission denied (13)

Who is in charge for fixing this?

Thanks,
Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On Thu, 2015-10-29 at 08:48 -0600, Ken Dreyer wrote:
> On Wed, Oct 28, 2015 at 7:54 PM, Matt Taylor 
> wrote:
> > I still see rsync errors due to permissions on the remote side:
> > 
> 
> Thanks for the heads' up; I bet another upload rsync process got
> interrupted there.
> 
> I've run the following to remove all the oddly-named RPM files:
> 
>   for f in $(locate *.rpm.* ) ; do rm -i $f; done
> 
> Please let us know if there are other problems like this.

Since a month the rsync is broken again.

———
rsync: send_files failed to open "/debian-jewel/db/lockfile" (in ceph):
Permission denied (13)
rsync: send_files failed to open "/debian-jewel/pool/main/c/ceph/.ceph-
fuse-dbg_10.1.0-1~bpo80+1_amd64.deb.h0JvHM" (in ceph): Permission
denied (13)
rsync: send_files failed to open "/debian-jewel/pool/main/c/ceph/.ceph-
test-dbg_10.1.0-1trusty_amd64.deb.06D0AZ" (in ceph): Permission denied
(13)
rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1655) [generator=3.1.1]
———

you fixed this problem last time, so please fix it again :-)

try: 
  find /path/to/mirror -iname '.*.deb.* -ls
if all files that are found are broken:
  find /path/to/mirror -iname '.*.deb.* -delete

Thanks a lot
Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] 10Gbit switch advice for small ceph cluster upgrade

[Björn Lässig]

On Fri, 2016-12-16 at 09:52 +0100, Robert Sander wrote:
> On 15.12.2016 16:49, Bjoern Laessig wrote:
> 
> > What does your Cluster do? Where is your data. What happens now?
> 
> You could configure the interfaces between the nodes as pointopoint
> links and run OSPF on them. The cluster nodes then would have their
> node IP on a dummy interface. OSPF would sort out the routing.

yes, this is worth a try! Thanks.
Bird should be suitable for me.

Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Ceph performances

[Björn Lässig]

On 11/07/2015 09:44 AM, Oliver Dzombic wrote:
> setting inode64 in osd_mount_options_xfs might help a little.

sorry, inode64 is the default mount option with xfs.

Björn

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On 10/27/2015 06:21 PM, Ken Dreyer wrote:
> Thanks, I've deleted it from the download.ceph.com web server.

Thanks a lot!
My mirror is up-to-date now.

Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

Hi,

after having some problems with ipv6 and download.ceph.com, i made a 
mirror (debian-hammer only) for my ipv6-only cluster.


Unfortunately after the release of 0.94.5 the rsync breaks with:

# less rsync-ftpsync-mirror_ceph.error.0
rsync: send_files failed to open 
"/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ" 
(in ceph): Permission denied (13)
rsync error: some files/attrs were not transferred (see previous errors) 
(code 23) at main.c(1655) [generator=3.1.1]


indeed there is:

[09:40:49] ~ > rsync -4 -L 
download.ceph.com::ceph/debian-hammer/pool/main/c/ceph/.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ
-rw--- 91,488,256 2015/10/26 19:36:46 
.ceph-dbg_0.94.5-1trusty_amd64.deb.3xQnIQ


i would be thankful, if you could remove this broken file or complete 
the mirror process.


Thanks in advance

Björn Lässig

PS: a debian like mirror-push mechanic via 
https://ftp-master.debian.org/git/archvsync.git would be very helpful. 
This would add traces and Archive-in-progress files and all the other 
things, the debian-mirror-admins had to fix over the years. This would 
lead to less traffic for you and always up-to-date mirrors for us.

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On 10/27/2015 10:22 AM, Wido den Hollander wrote:

On 27-10-15 09:51, Björn Lässig wrote:

after having some problems with ipv6 and download.ceph.com, i made a
mirror (debian-hammer only) for my ipv6-only cluster.


I see you are from Germany, you can also sync from eu.ceph.com


good to know, that you give rsync access, too.
The Problem here is that there is no indication, that the primary mirror 
(download.ceph.com) is in a complete state. I suppose you you use some 
sort of cron-method for synchronizing with download.c.c. So in case the 
primary ist broken, the chain needs more time to repair.
Thats why i decided it is more robust to rsync from download.c.c 
directly. i will change this, if eu.c.c is pushed on update by download.c.c



PS: a debian like mirror-push mechanic via
https://ftp-master.debian.org/git/archvsync.git would be very helpful.
This would add traces and Archive-in-progress files and all the other
things, the debian-mirror-admins had to fix over the years. This would
lead to less traffic for you and always up-to-date mirrors for us.


That is still on my TODO list. We need a nice mirror system for Ceph.


i run a secondary debian mirror at my old university for more than 10 
years now. If you start working on that, please keep me in the loop.
Maybe starting with a ceph-mirror mailinglist would be a possibility, to 
concentrate all interested people.


Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] rsync mirror download.ceph.com - broken file on rsync server

[Björn Lässig]

On 10/27/2015 02:50 PM, Wido den Hollander wrote:

i run a secondary debian mirror at my old university for more than 10
years now. If you start working on that, please keep me in the loop.
Maybe starting with a ceph-mirror mailinglist would be a possibility, to
concentrate all interested people.


Yes, that would be an idea. I can ask for that mailinglist.

But keep in mind, download.ceph.com doesn't only serve DEB, it also
serves tarballs and RPM packages.


yes. thats why we would need help from some fedora/redhat/centos admins. 
I do not know how their stuff is distributed. i am a debian guy since i 
crushed yast a very long time ago.


I see 3 points:
 * give clusteradmins an easy way to get a OS/ceph-version mirror for
   their architecture (this is my use case)
 * doing a full mirror (eu.ceph.com)
 * inform mirror-owners (push mirrors), if something changed

Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph-hammer and debian jessie - missing files on repository

[Björn Lässig]

On 10/21/2015 08:50 PM, Alfredo Deza wrote:

This shouldn't be a problem, would you mind trying again? I just
managed to install on Debian Jessie without problems


after syncing our mirror via IPv4 again (ipv6 is still broken), we have 
the missing bpo80 packages for debian-jessie!


Thanks for your help

Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] ceph-hammer and debian jessie - missing files on repository

[Björn Lässig]

Hi,

Thanks guys for supporting the latest debian stable release with latest 
ceph stable!


as version 0.94.4. has been released, i tried to upgrade my 
debian/jessie cluster with hammer/wheezy packages to hammer/jessie.


Unfortunately the download.ceph.com/debian-hammer debian repository is 
in some strange state. (even, if you ignore, that the ipv6 connection is 
very flaky from different sites in europe)


eg:

in
http://download.ceph.com/debian-hammer/dists/jessie/main/binary-amd64/Packages
a package called ceph-common is referenced

- /
Package: ceph-common
Version: 0.94.4-1~bpo80+1
Architecture: amd64
Filename: pool/main/c/ceph/ceph-common_0.94.4-1~bpo80+1_amd64.deb
--\

but this package file does not exists. All files whith version ~bpo80 do 
not exist. (yet?)



could you check this please?

Thanks in advance
Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] download.ceph.com unreachable IPv6 [was: v9.1.0 Infernalis release candidate released]

[Björn Lässig]

On 10/15/2015 09:40 PM, Ken Dreyer wrote:

I wonder if there are routing issues with IPv6 in the DreamHost cloud?

http://ipv6-test.com/validate.php prints the right IP, but then "web
server is unreachable : Connection timed out"


Getting the same error here.
With sixxs 4 out of 5 wgets are failing. ping6 is dropped.

We tried from different sites in .at .uk and .de. Tries from uk and de 
are failing mostly. at is working.


does ''ping6 download.ceph.com'' works for you?

regards
  Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] download.ceph.com unreachable IPv6 [was: v9.1.0 Infernalis release candidate released]

[Björn Lässig]

On 10/14/2015 05:11 PM, Wido den Hollander wrote:
> 
> 
> On 14-10-15 16:30, Björn Lässig wrote:
>> On 10/13/2015 11:01 PM, Sage Weil wrote:
>>>   http://download.ceph.com/debian-testing
>>
>> unfortunately this site is not reachable at the moment.
>>
> wido@wido-desktop:~$ wget -6
> http://download.ceph.com/debian-testing/dists/wheezy/InRelease
> […]
> 2015-10-14 17:10:41 (306 MB/s) - ‘InRelease’ saved [6873/6873]

We tried from different locations with

 * sixxs
 * our AS (AS201824)
 * some testing from hetzner. (AS24940)

On some locations ''wget -6
http://download.ceph.com/debian-testing/dists/wheezy/InRelease'' does
not work, on some of them, it only works 1 out of 5 times.

Testing is hard. They drop icmp6-ping. I have no idea, why they are
doing this.

Their PTR record for download.ceph.com is missing, but thats not the
point here.


regards,

 Björn Lässig

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] [sepia] debian jessie repository ?

[Björn Lässig]

On 09/25/2015 03:10 PM, Jogi Hofmüller wrote:

Am 2015-09-11 um 13:20 schrieb Florent B:


Jessie repository will be available on next Hammer release ;)


An how should I continue installing ceph meanwhile?  ceph-deploy new ...
overwrites the /etc/apt/sources.list.d/ceph.list and hence throws an
error :(


I did intalling hammer on jessie a couple of weeks ago:

add
deb http://ceph.com/debian-hammer/ wheezy main

and add some debian-wheezy sources to your sources.list

do the ''ceph-deploy install'' step by hand. It is crap, but it works 
with aptitude in interactive mode. search for ceph - setup the hammer 
version and then solve the dependencies manually.
For the first time it takes 15 minutes, every repetition again 5 
minutes. If your cluster has 100 nodes ... do not try this unless you 
are bored to death ;-(


Björn
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph cluster_network with linklocal ipv6

[Björn Lässig]

On 08/18/2015 03:39 PM, Björn Lässig wrote:
 For not having any dependencies in my cluster network, i want to use
 only ipv6 link-local addresses on interface 'cephnet'.
 cluster_network = fe80::%cephnet/64

RFC4007 11.7

   The IPv6 addressing architecture [1] also defines the syntax of IPv6
   prefixes.  If the address portion of a prefix is non-global and its
   scope zone should be disambiguated, the address portion SHOULD be in
   the format.  For example, a link-local prefix fe80::/64 on the second
   link can be represented as follows:

fe80::%2/64

   In this combination, it is important to place the zone index portion
   before the prefix length when we consider parsing the format by a
   name-to-address library function [11].  That is, we can first
   separate the address with the zone index from the prefix length, and
   just pass the former to the library function.

so my original format would be the correct one.

just looked into the code ... and this is exactly what happens in:

src/common/ipaddr.cc: bool parse_network(const char *s, […]) {

it looks where a '/' is and try to cast everything after to a longint.
Everything before the '/' is thrown into inet_pton

ok = inet_pton(AF_INET6, addr, ((struct sockaddr_in6*)network)-sin6_addr);

and inet_pton6 from glibc does not look at '%'.
No chance, this code could work with an interfacename after or before
the '/'

I'm using addresses with global scope now.

Björn



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ceph cluster_network with linklocal ipv6

[Björn Lässig]

On 08/18/2015 04:32 PM, Jan Schermer wrote:

Should ceph care about what scope the address is in? We don't specify it for 
ipv4 anyway, or is link-scope special in some way?


fe80::/64 is on every ipv6 enabled interface ... thats different from 
legacy ip.



And isn't this the correct syntax actually?

cluster_network = fe80::/64%cephnet


That is a very good question! I will look into it.

Björn

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com