Re: [ceph-users] Luminous Upgrade KRBD

[Ashley Merrick]

Okie thanks all, will hold off 

-Original Message-
From: Ilya Dryomov [mailto:idryo...@gmail.com] 
Sent: 06 September 2017 17:58
To: Ashley Merrick <ash...@amerrick.co.uk>
Cc: Henrik Korkuc <li...@kirneh.eu>; ceph-us...@ceph.com
Subject: Re: [ceph-users] Luminous Upgrade KRBD

On Wed, Sep 6, 2017 at 11:23 AM, Ashley Merrick <ash...@amerrick.co.uk> wrote:
> Only drive for it was to be able to use this:
>
> http://docs.ceph.com/docs/master/rados/operations/upmap/
>
> To see if would help with the current very uneven PG MAP across 100+ OSD's, 
> something that can wait if current kernel isn't ready.

I guess it depends on the meaning of "current".  Proxmox's latest is indeed 
4.10.  The just-released upstream kernel 4.13 does support upmap exception 
table and other luminous features.

That said, the balancer isn't fully baked yet, so I'd wait.

Thanks,

Ilya
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[Ilya Dryomov]

On Wed, Sep 6, 2017 at 11:23 AM, Ashley Merrick  wrote:
> Only drive for it was to be able to use this:
>
> http://docs.ceph.com/docs/master/rados/operations/upmap/
>
> To see if would help with the current very uneven PG MAP across 100+ OSD's, 
> something that can wait if current kernel isn't ready.

I guess it depends on the meaning of "current".  Proxmox's latest is
indeed 4.10.  The just-released upstream kernel 4.13 does support upmap
exception table and other luminous features.

That said, the balancer isn't fully baked yet, so I'd wait.

Thanks,

Ilya
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[Ashley Merrick]

Only drive for it was to be able to use this:

http://docs.ceph.com/docs/master/rados/operations/upmap/

To see if would help with the current very uneven PG MAP across 100+ OSD's, 
something that can wait if current kernel isn't ready.

,Ashley

-Original Message-
From: Ilya Dryomov [mailto:idryo...@gmail.com] 
Sent: 06 September 2017 17:09
To: Henrik Korkuc <li...@kirneh.eu>
Cc: Ashley Merrick <ash...@amerrick.co.uk>; ceph-us...@ceph.com
Subject: Re: [ceph-users] Luminous Upgrade KRBD

On Wed, Sep 6, 2017 at 9:16 AM, Henrik Korkuc <li...@kirneh.eu> wrote:
> On 17-09-06 09:10, Ashley Merrick wrote:
>
> I was just going by : 
> docs.ceph.com/docs/master/start/os-recommendations/
>
>
> Which states 4.9
>
>
> docs.ceph.com/docs/master/rados/operations/crush-map
>
>
> Only goes as far as Jewel and states 4.5
>
>
> Not sure where else I can find a concrete answer to if 4.10 is new enough.
>
> Well, it looks like docs may need to be revisited as I was unable to 
> use kcephfs on 4.9 with luminous before downgrading tunables, not sure 
> about 4.10.
>
>
>
> ,Ashley
>
> 
> From: Henrik Korkuc <li...@kirneh.eu>
> Sent: 06 September 2017 06:58:52
> To: Ashley Merrick; ceph-us...@ceph.com
> Subject: Re: [ceph-users] Luminous Upgrade KRBD
>
> On 17-09-06 07:33, Ashley Merrick wrote:
>
> Hello,
>
> Have recently upgraded a cluster to Luminous (Running Proxmox), at the 
> same time I have upgraded the Compute Cluster to 5.x meaning we now 
> run the latest kernel version (Linux 4.10.15-1) Looking to do the following :
>
> ceph osd set-require-min-compat-client luminous

This would effectively require 4.13 kernel for krbd and kcephfs.

You won't gain much by requiring luminous clients, so unless you have a 
well-formulated reason to do it, I'd advise against it.  That's also the reason 
os-recommendations page still lists 4.9 -- it is the latest jewel-compatible 
LTS kernel.

Thanks,

Ilya
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[ceph]

Quick drop-in, if this is a suitable solution: rbd-nbd
This will give you, for a small performance cost, a block device using
librbd (in userspace)

On 06/09/2017 11:08, Ilya Dryomov wrote:
> On Wed, Sep 6, 2017 at 9:16 AM, Henrik Korkuc <li...@kirneh.eu> wrote:
>> On 17-09-06 09:10, Ashley Merrick wrote:
>>
>> I was just going by : docs.ceph.com/docs/master/start/os-recommendations/
>>
>>
>> Which states 4.9
>>
>>
>> docs.ceph.com/docs/master/rados/operations/crush-map
>>
>>
>> Only goes as far as Jewel and states 4.5
>>
>>
>> Not sure where else I can find a concrete answer to if 4.10 is new enough.
>>
>> Well, it looks like docs may need to be revisited as I was unable to use
>> kcephfs on 4.9 with luminous before downgrading tunables, not sure about
>> 4.10.
>>
>>
>>
>> ,Ashley
>>
>> ____________
>> From: Henrik Korkuc <li...@kirneh.eu>
>> Sent: 06 September 2017 06:58:52
>> To: Ashley Merrick; ceph-us...@ceph.com
>> Subject: Re: [ceph-users] Luminous Upgrade KRBD
>>
>> On 17-09-06 07:33, Ashley Merrick wrote:
>>
>> Hello,
>>
>> Have recently upgraded a cluster to Luminous (Running Proxmox), at the same
>> time I have upgraded the Compute Cluster to 5.x meaning we now run the
>> latest kernel version (Linux 4.10.15-1) Looking to do the following :
>>
>> ceph osd set-require-min-compat-client luminous
> 
> This would effectively require 4.13 kernel for krbd and kcephfs.
> 
> You won't gain much by requiring luminous clients, so unless you have
> a well-formulated reason to do it, I'd advise against it.  That's also
> the reason os-recommendations page still lists 4.9 -- it is the latest
> jewel-compatible LTS kernel.
> 
> Thanks,
> 
> Ilya
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[Ilya Dryomov]

On Wed, Sep 6, 2017 at 9:16 AM, Henrik Korkuc <li...@kirneh.eu> wrote:
> On 17-09-06 09:10, Ashley Merrick wrote:
>
> I was just going by : docs.ceph.com/docs/master/start/os-recommendations/
>
>
> Which states 4.9
>
>
> docs.ceph.com/docs/master/rados/operations/crush-map
>
>
> Only goes as far as Jewel and states 4.5
>
>
> Not sure where else I can find a concrete answer to if 4.10 is new enough.
>
> Well, it looks like docs may need to be revisited as I was unable to use
> kcephfs on 4.9 with luminous before downgrading tunables, not sure about
> 4.10.
>
>
>
> ,Ashley
>
> 
> From: Henrik Korkuc <li...@kirneh.eu>
> Sent: 06 September 2017 06:58:52
> To: Ashley Merrick; ceph-us...@ceph.com
> Subject: Re: [ceph-users] Luminous Upgrade KRBD
>
> On 17-09-06 07:33, Ashley Merrick wrote:
>
> Hello,
>
> Have recently upgraded a cluster to Luminous (Running Proxmox), at the same
> time I have upgraded the Compute Cluster to 5.x meaning we now run the
> latest kernel version (Linux 4.10.15-1) Looking to do the following :
>
> ceph osd set-require-min-compat-client luminous

This would effectively require 4.13 kernel for krbd and kcephfs.

You won't gain much by requiring luminous clients, so unless you have
a well-formulated reason to do it, I'd advise against it.  That's also
the reason os-recommendations page still lists 4.9 -- it is the latest
jewel-compatible LTS kernel.

Thanks,

Ilya
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[Henrik Korkuc]

On 17-09-06 09:10, Ashley Merrick wrote:


I was just going by : docs.ceph.com/docs/master/start/os-recommendations/


Which states 4.9


docs.ceph.com/docs/master/rados/operations/crush-map


Only goes as far as Jewel and states 4.5


Not sure where else I can find a concrete answer to if 4.10 is new enough.

Well, it looks like docs may need to be revisited as I was unable to use 
kcephfs on 4.9 with luminous before downgrading tunables, not sure about 
4.10.




,Ashley


*From:* Henrik Korkuc <li...@kirneh.eu>
*Sent:* 06 September 2017 06:58:52
*To:* Ashley Merrick; ceph-us...@ceph.com
*Subject:* Re: [ceph-users] Luminous Upgrade KRBD
On 17-09-06 07:33, Ashley Merrick wrote:

Hello,

Have recently upgraded a cluster to Luminous (Running Proxmox), at 
the same time I have upgraded the Compute Cluster to 5.x meaning we 
now run the latest kernel version (Linux 4.10.15-1) Looking to do the 
following :


ceph osd set-require-min-compat-client luminous

Does 4.10 kernel support luminous features? I am afraid (but do not 
have info to back it up) that 4.10 is too old for Luminous features.


Below is the output of ceph features, the 4 number next to the last 
row of luminous is as expected for the 4 Compute nodes, are the other 
4 spread across hammer & jewel log's of when the node's last 
connected before they was upgraded to Proxmox 5.0, am I safe to run 
the above command? No other RBD resources are connected to this cluster.


  "client": {
        "group": {
            "features": "0x106b84a842a42",
            "release": "hammer",
            "num": 1
        },
        "group": {
            "features": "0x40106b84a842a52",
            "release": "jewel",
            "num": 3
        },
        "group": {
            "features": "0x1ffddff8eea4fffb",
            "release": "luminous",
            "num": 4
        }




___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com





___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[Ashley Merrick]

I was just going by : docs.ceph.com/docs/master/start/os-recommendations/


Which states 4.9


docs.ceph.com/docs/master/rados/operations/crush-map


Only goes as far as Jewel and states 4.5


Not sure where else I can find a concrete answer to if 4.10 is new enough.


,Ashley


From: Henrik Korkuc <li...@kirneh.eu>
Sent: 06 September 2017 06:58:52
To: Ashley Merrick; ceph-us...@ceph.com
Subject: Re: [ceph-users] Luminous Upgrade KRBD

On 17-09-06 07:33, Ashley Merrick wrote:
Hello,

Have recently upgraded a cluster to Luminous (Running Proxmox), at the same 
time I have upgraded the Compute Cluster to 5.x meaning we now run the latest 
kernel version (Linux 4.10.15-1) Looking to do the following :

ceph osd set-require-min-compat-client luminous

Does 4.10 kernel support luminous features? I am afraid (but do not have info 
to back it up) that 4.10 is too old for Luminous features.

Below is the output of ceph features, the 4 number next to the last row of 
luminous is as expected for the 4 Compute nodes, are the other 4 spread across 
hammer & jewel log's of when the node's last connected before they was upgraded 
to Proxmox 5.0, am I safe to run the above command? No other RBD resources are 
connected to this cluster.

  "client": {
"group": {
"features": "0x106b84a842a42",
"release": "hammer",
"num": 1
},
"group": {
"features": "0x40106b84a842a52",
"release": "jewel",
"num": 3
},
"group": {
"features": "0x1ffddff8eea4fffb",
"release": "luminous",
"num": 4
}





___
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous Upgrade KRBD

[Henrik Korkuc]

On 17-09-06 07:33, Ashley Merrick wrote:

Hello,

Have recently upgraded a cluster to Luminous (Running Proxmox), at the 
same time I have upgraded the Compute Cluster to 5.x meaning we now 
run the latest kernel version (Linux 4.10.15-1) Looking to do the 
following :


ceph osd set-require-min-compat-client luminous

Does 4.10 kernel support luminous features? I am afraid (but do not have 
info to back it up) that 4.10 is too old for Luminous features.


Below is the output of ceph features, the 4 number next to the last 
row of luminous is as expected for the 4 Compute nodes, are the other 
4 spread across hammer & jewel log's of when the node's last connected 
before they was upgraded to Proxmox 5.0, am I safe to run the above 
command? No other RBD resources are connected to this cluster.


  "client": {
        "group": {
            "features": "0x106b84a842a42",
            "release": "hammer",
            "num": 1
        },
        "group": {
            "features": "0x40106b84a842a52",
            "release": "jewel",
            "num": 3
        },
        "group": {
            "features": "0x1ffddff8eea4fffb",
            "release": "luminous",
            "num": 4
        }




___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com