Re: [ceph-users] Networking/naming doubt

[Oscar Segarra]

Hi David,

Thanks a lot for your comments!

I just want to utilize a different network than the public one (where dns
resolves the name) for ceph-deploy and client connections.

For example with 3 nics:

Nic1: Public (internet acces)
Nic2: Ceph-mon (clients and ceph-deploy)
Nic3: Ceph-osd

Thanks a lot for your help!

El 28 jul. 2017 2:25 a. m., "David Turner"  escribió:

The only thing that is supposed to use the cluster network are the OSDs.
Not even the MONs access the cluster network. I am sure that if you have a
need to make this work that you can find a way, but I don't know that one
exists in the standard tool set.

You might try temporarily setting the /etc/hosts reference for vdicnode02
and vdicnode03 to the cluster network and use the proper hosts name in the
ceph-deploy command. Ceph cluster operations do not use dns at all, so you
could probably leave your /etc/hosts in this state. I don't know if it
would work though. It's really not intended for any communication to happen
on this subnet other than inter-OSD traffic.



On Thu, Jul 27, 2017 at 6:31 PM Oscar Segarra 
wrote:

> Sorry! I'd like to add that I want to use the cluster network for both
> purposes:
>
> ceph-deploy --username vdicceph new vdicnode01 --cluster-network
> 192.168.100.0/24 --public-network 192.168.100.0/24
>
> Thanks a lot
>
>
> 2017-07-28 0:29 GMT+02:00 Oscar Segarra :
>
>> Hi,
>>
>> ¿Do you mean that for security reasons ceph-deploy can only be executed
>> from the public interface?
>>
>> Looks extrange that one cannot decide what network use for ceph-deploy...
>> I could have a dedicated network for ceph-deploy... :S
>>
>> Thanks a lot
>>
>> 2017-07-28 0:03 GMT+02:00 Roger Brown :
>>
>>> I could be wrong, but I think you cannot achieve this objective. If you
>>> declare a cluster network, OSDs will route heartbeat, object replication
>>> and recovery traffic over the cluster network. We prefer that the cluster
>>> network is NOT reachable from the public network or the Internet for added
>>> security. Therefore it will not work with ceph-deploy actions.
>>> Source: http://docs.ceph.com/docs/master/rados/
>>> configuration/network-config-ref/
>>>
>>>
>>> On Thu, Jul 27, 2017 at 3:53 PM Oscar Segarra 
>>> wrote:
>>>
 Hi,

 In my environment I have 3 hosts, every host has 2 network interfaces:

 public: 192.168.2.0/24
 cluster: 192.168.100.0/24

 The hostname "vdicnode01", "vdicnode02" and "vdicnode03" are resolved
 by public DNS through the public interface, that means the "ping
 vdicnode01" will resolve 192.168.2.1.

 In my environment the "admin" node is the first node vdicnode01 and I'd
 like all the deployment "ceph-deploy" and all osd traffic to go from the
 cluster network.

 1) To begin with, I create the cluster and I want all traffic to go
 from the cluster network:
 ceph-deploy --username vdicceph new vdicnode01 --cluster-network
 192.168.100.0/24 --public-network 192.168.100.0/24

 2) The problem comes when I have to launch my commands to the other
 hosts for example, from node vdicnode01 I execute:

 2.1) ceph-deploy --username vdicceph osd create vdicnode02:sdb
 --> Finishes Ok but communication goes through the public interface

 2.2) ceph-deploy --username vdicceph osd create vdicnode02.local:sdb
 --> vdicnode02.local is added manually in /etc/hosts (assigned a
 cluster IP)
 --> It raises some errors/warnings becase vdicnod02.local is not the
 real hostname. Some files are created with vdicnode02.local in the middle
 of the name of the file and some errors appear when starting up the osd
 service related to "file does not exist"

 2.3) ceph-deploy --username vdicceph osd create vdicnode02-priv:sdb
 --> vdicnode02-priv is added manually in /etc/hosts (assigned a cluster
 IP)
 --> It raises some errors/warnings becase vdicnod02-priv is not the
 real hostname. Some files are created with vdicnode02-priv in the middle of
 the name of the file and some errors appear when starting up the osd
 service related to "file does not exist"

 What would be the right way to achieve my objective?

 If is there any documentation I have not found, please redirect me...

 Thanks a lot for your help in advance.
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

>>>
>>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Networking/naming doubt

[David Turner]

The only thing that is supposed to use the cluster network are the OSDs.
Not even the MONs access the cluster network. I am sure that if you have a
need to make this work that you can find a way, but I don't know that one
exists in the standard tool set.

You might try temporarily setting the /etc/hosts reference for vdicnode02
and vdicnode03 to the cluster network and use the proper hosts name in the
ceph-deploy command. Ceph cluster operations do not use dns at all, so you
could probably leave your /etc/hosts in this state. I don't know if it
would work though. It's really not intended for any communication to happen
on this subnet other than inter-OSD traffic.



On Thu, Jul 27, 2017 at 6:31 PM Oscar Segarra 
wrote:

> Sorry! I'd like to add that I want to use the cluster network for both
> purposes:
>
> ceph-deploy --username vdicceph new vdicnode01 --cluster-network
> 192.168.100.0/24 --public-network 192.168.100.0/24
>
> Thanks a lot
>
>
> 2017-07-28 0:29 GMT+02:00 Oscar Segarra :
>
>> Hi,
>>
>> ¿Do you mean that for security reasons ceph-deploy can only be executed
>> from the public interface?
>>
>> Looks extrange that one cannot decide what network use for ceph-deploy...
>> I could have a dedicated network for ceph-deploy... :S
>>
>> Thanks a lot
>>
>> 2017-07-28 0:03 GMT+02:00 Roger Brown :
>>
>>> I could be wrong, but I think you cannot achieve this objective. If you
>>> declare a cluster network, OSDs will route heartbeat, object replication
>>> and recovery traffic over the cluster network. We prefer that the cluster
>>> network is NOT reachable from the public network or the Internet for added
>>> security. Therefore it will not work with ceph-deploy actions.
>>> Source:
>>> http://docs.ceph.com/docs/master/rados/configuration/network-config-ref/
>>>
>>>
>>> On Thu, Jul 27, 2017 at 3:53 PM Oscar Segarra 
>>> wrote:
>>>
 Hi,

 In my environment I have 3 hosts, every host has 2 network interfaces:

 public: 192.168.2.0/24
 cluster: 192.168.100.0/24

 The hostname "vdicnode01", "vdicnode02" and "vdicnode03" are resolved
 by public DNS through the public interface, that means the "ping
 vdicnode01" will resolve 192.168.2.1.

 In my environment the "admin" node is the first node vdicnode01 and I'd
 like all the deployment "ceph-deploy" and all osd traffic to go from the
 cluster network.

 1) To begin with, I create the cluster and I want all traffic to go
 from the cluster network:
 ceph-deploy --username vdicceph new vdicnode01 --cluster-network
 192.168.100.0/24 --public-network 192.168.100.0/24

 2) The problem comes when I have to launch my commands to the other
 hosts for example, from node vdicnode01 I execute:

 2.1) ceph-deploy --username vdicceph osd create vdicnode02:sdb
 --> Finishes Ok but communication goes through the public interface

 2.2) ceph-deploy --username vdicceph osd create vdicnode02.local:sdb
 --> vdicnode02.local is added manually in /etc/hosts (assigned a
 cluster IP)
 --> It raises some errors/warnings becase vdicnod02.local is not the
 real hostname. Some files are created with vdicnode02.local in the middle
 of the name of the file and some errors appear when starting up the osd
 service related to "file does not exist"

 2.3) ceph-deploy --username vdicceph osd create vdicnode02-priv:sdb
 --> vdicnode02-priv is added manually in /etc/hosts (assigned a cluster
 IP)
 --> It raises some errors/warnings becase vdicnod02-priv is not the
 real hostname. Some files are created with vdicnode02-priv in the middle of
 the name of the file and some errors appear when starting up the osd
 service related to "file does not exist"

 What would be the right way to achieve my objective?

 If is there any documentation I have not found, please redirect me...

 Thanks a lot for your help in advance.
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

>>>
>>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Networking/naming doubt

[Oscar Segarra]

Sorry! I'd like to add that I want to use the cluster network for both
purposes:

ceph-deploy --username vdicceph new vdicnode01 --cluster-network
192.168.100.0/24 --public-network 192.168.100.0/24

Thanks a lot

2017-07-28 0:29 GMT+02:00 Oscar Segarra :

> Hi,
>
> ¿Do you mean that for security reasons ceph-deploy can only be executed
> from the public interface?
>
> Looks extrange that one cannot decide what network use for ceph-deploy...
> I could have a dedicated network for ceph-deploy... :S
>
> Thanks a lot
>
> 2017-07-28 0:03 GMT+02:00 Roger Brown :
>
>> I could be wrong, but I think you cannot achieve this objective. If you
>> declare a cluster network, OSDs will route heartbeat, object replication
>> and recovery traffic over the cluster network. We prefer that the cluster
>> network is NOT reachable from the public network or the Internet for added
>> security. Therefore it will not work with ceph-deploy actions.
>> Source: http://docs.ceph.com/docs/master/rados/configuration
>> /network-config-ref/
>>
>>
>> On Thu, Jul 27, 2017 at 3:53 PM Oscar Segarra 
>> wrote:
>>
>>> Hi,
>>>
>>> In my environment I have 3 hosts, every host has 2 network interfaces:
>>>
>>> public: 192.168.2.0/24
>>> cluster: 192.168.100.0/24
>>>
>>> The hostname "vdicnode01", "vdicnode02" and "vdicnode03" are resolved by
>>> public DNS through the public interface, that means the "ping vdicnode01"
>>> will resolve 192.168.2.1.
>>>
>>> In my environment the "admin" node is the first node vdicnode01 and I'd
>>> like all the deployment "ceph-deploy" and all osd traffic to go from the
>>> cluster network.
>>>
>>> 1) To begin with, I create the cluster and I want all traffic to go from
>>> the cluster network:
>>> ceph-deploy --username vdicceph new vdicnode01 --cluster-network
>>> 192.168.100.0/24 --public-network 192.168.100.0/24
>>>
>>> 2) The problem comes when I have to launch my commands to the other
>>> hosts for example, from node vdicnode01 I execute:
>>>
>>> 2.1) ceph-deploy --username vdicceph osd create vdicnode02:sdb
>>> --> Finishes Ok but communication goes through the public interface
>>>
>>> 2.2) ceph-deploy --username vdicceph osd create vdicnode02.local:sdb
>>> --> vdicnode02.local is added manually in /etc/hosts (assigned a cluster
>>> IP)
>>> --> It raises some errors/warnings becase vdicnod02.local is not the
>>> real hostname. Some files are created with vdicnode02.local in the middle
>>> of the name of the file and some errors appear when starting up the osd
>>> service related to "file does not exist"
>>>
>>> 2.3) ceph-deploy --username vdicceph osd create vdicnode02-priv:sdb
>>> --> vdicnode02-priv is added manually in /etc/hosts (assigned a cluster
>>> IP)
>>> --> It raises some errors/warnings becase vdicnod02-priv is not the real
>>> hostname. Some files are created with vdicnode02-priv in the middle of the
>>> name of the file and some errors appear when starting up the osd service
>>> related to "file does not exist"
>>>
>>> What would be the right way to achieve my objective?
>>>
>>> If is there any documentation I have not found, please redirect me...
>>>
>>> Thanks a lot for your help in advance.
>>> ___
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Networking/naming doubt

[Roger Brown]

I could be wrong, but I think you cannot achieve this objective. If you
declare a cluster network, OSDs will route heartbeat, object replication
and recovery traffic over the cluster network. We prefer that the cluster
network is NOT reachable from the public network or the Internet for added
security. Therefore it will not work with ceph-deploy actions.
Source:
http://docs.ceph.com/docs/master/rados/configuration/network-config-ref/


On Thu, Jul 27, 2017 at 3:53 PM Oscar Segarra 
wrote:

> Hi,
>
> In my environment I have 3 hosts, every host has 2 network interfaces:
>
> public: 192.168.2.0/24
> cluster: 192.168.100.0/24
>
> The hostname "vdicnode01", "vdicnode02" and "vdicnode03" are resolved by
> public DNS through the public interface, that means the "ping vdicnode01"
> will resolve 192.168.2.1.
>
> In my environment the "admin" node is the first node vdicnode01 and I'd
> like all the deployment "ceph-deploy" and all osd traffic to go from the
> cluster network.
>
> 1) To begin with, I create the cluster and I want all traffic to go from
> the cluster network:
> ceph-deploy --username vdicceph new vdicnode01 --cluster-network
> 192.168.100.0/24 --public-network 192.168.100.0/24
>
> 2) The problem comes when I have to launch my commands to the other hosts
> for example, from node vdicnode01 I execute:
>
> 2.1) ceph-deploy --username vdicceph osd create vdicnode02:sdb
> --> Finishes Ok but communication goes through the public interface
>
> 2.2) ceph-deploy --username vdicceph osd create vdicnode02.local:sdb
> --> vdicnode02.local is added manually in /etc/hosts (assigned a cluster
> IP)
> --> It raises some errors/warnings becase vdicnod02.local is not the real
> hostname. Some files are created with vdicnode02.local in the middle of the
> name of the file and some errors appear when starting up the osd service
> related to "file does not exist"
>
> 2.3) ceph-deploy --username vdicceph osd create vdicnode02-priv:sdb
> --> vdicnode02-priv is added manually in /etc/hosts (assigned a cluster IP)
> --> It raises some errors/warnings becase vdicnod02-priv is not the real
> hostname. Some files are created with vdicnode02-priv in the middle of the
> name of the file and some errors appear when starting up the osd service
> related to "file does not exist"
>
> What would be the right way to achieve my objective?
>
> If is there any documentation I have not found, please redirect me...
>
> Thanks a lot for your help in advance.
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Networking/naming doubt

[Oscar Segarra]

Hi,

In my environment I have 3 hosts, every host has 2 network interfaces:

public: 192.168.2.0/24
cluster: 192.168.100.0/24

The hostname "vdicnode01", "vdicnode02" and "vdicnode03" are resolved by
public DNS through the public interface, that means the "ping vdicnode01"
will resolve 192.168.2.1.

In my environment the "admin" node is the first node vdicnode01 and I'd
like all the deployment "ceph-deploy" and all osd traffic to go from the
cluster network.

1) To begin with, I create the cluster and I want all traffic to go from
the cluster network:
ceph-deploy --username vdicceph new vdicnode01 --cluster-network
192.168.100.0/24 --public-network 192.168.100.0/24

2) The problem comes when I have to launch my commands to the other hosts
for example, from node vdicnode01 I execute:

2.1) ceph-deploy --username vdicceph osd create vdicnode02:sdb
--> Finishes Ok but communication goes through the public interface

2.2) ceph-deploy --username vdicceph osd create vdicnode02.local:sdb
--> vdicnode02.local is added manually in /etc/hosts (assigned a cluster IP)
--> It raises some errors/warnings becase vdicnod02.local is not the real
hostname. Some files are created with vdicnode02.local in the middle of the
name of the file and some errors appear when starting up the osd service
related to "file does not exist"

2.3) ceph-deploy --username vdicceph osd create vdicnode02-priv:sdb
--> vdicnode02-priv is added manually in /etc/hosts (assigned a cluster IP)
--> It raises some errors/warnings becase vdicnod02-priv is not the real
hostname. Some files are created with vdicnode02-priv in the middle of the
name of the file and some errors appear when starting up the osd service
related to "file does not exist"

What would be the right way to achieve my objective?

If is there any documentation I have not found, please redirect me...

Thanks a lot for your help in advance.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com