[ceph-users] Re: Nautilus CentOS-7 rpm dependencies

[Fabrice Bacchella]

I had a similar problem with pacific when using the build from Centos, I switch 
to the rpm directly from ceph and it went fine.

> Le 31 mai 2021 à 10:29, Andreas Haupt  a écrit :
> 
> Dear all,
> 
> ceph-mgr-dashboard-15.2.13-0.el7.noarch contains three rpm dependencies
> that cannot be resolved here (not part of CentOS & EPEL 7):
> 
> python3-cherrypy
> python3-routes
> python3-jwt
> 
> Does anybody know where they are expected to come from?
> 
> Thanks,
> Andreas
> -- 
> | Andreas Haupt| E-Mail: andreas.ha...@desy.de
> |  DESY Zeuthen| WWW:http://www-zeuthen.desy.de/~ahaupt
> |  Platanenallee 6 | Phone:  +49/33762/7-7359
> |  D-15738 Zeuthen | Fax:+49/33762/7-7216
> 
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] logrotation in ceph 16.2.4

[Fabrice Bacchella]

I have a ceph cluster with 6 OSD servers.

2 are running 16.2.4 and logrotate failed with this message:
etc/cron.daily/logrotate:

error: Compressing program wrote following message to stderr when compressing 
log /var/log/ceph/ceph-osd.37.log-20210518:
gzip: stdin: file size changed while zipping

The four 4 16.2.3 OSD servers went fine.

The /etc/logrotate.d/ceph file is the same between versions. I think there 
might be a bug with the handling of signal SIGHUP (-1)
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] dashboard connecting to the object gateway

[Fabrice Bacchella]

I'm still trying to understand how the manager and dashboard connect to 
different object gateway, and I don’t really understand how it works.

Initially, I wanted to have each gateway listen only to localhost, on http:
[client.radosgw.<%= $id%>]
  rgw_frontends = beast endpoint=127.0.0.1:9080

It fails, but in a strange way. It was indeed connecting to the port 9080, so 
the mgr reads that line, but using the host IP.

I can easily understand why binding only to 127.0.0.1 was a bad idea, but I 
don't understand how it choose the IP to connect to.
When looking at the configuration using the dasbhoard, I see a field called 
hostname. This value seems to be read-only, as set it it in ceph.conf to a 
obvious wrong value (www.google.com ) does not change 
it.
Is it that field that is used ? It's important because if I want to have SSL 
verification, I must know the value: IP, hostname, FQDN ? that I will put in 
the Subject Alternative Name of the certificate.

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Certificat format for the SSL dashboard

[Fabrice Bacchella]

Thanks, my work around was to that problem was to put an Apache in front of the 
dashboard.

> Le 4 mai 2021 à 20:49, Ernesto Puerta  a écrit :
> 
> Hi Fabrice,
> 
> Don't worry, it has nothing to do with the internal format of those
> certificates. It's been a recent breakage and will be fixed in 16.2.2. In
> the meantime, you can find a workaround (well, you already nailed it: the
> CLI command that failed is just a wrapper around the "ceph config-key set"
> command), as well as the details and the PR here: the fix:
> https://tracker.ceph.com/issues/50519
> 
> 
> Kind Regards,
> Ernesto
> 
> 
> On Tue, May 4, 2021 at 12:10 PM Fabrice Bacchella <
> fabrice.bacche...@orange.fr> wrote:
> 
>> And worst:
>> 
>> $ ceph config-key set mgr/restful/fa42/crt -i /data/ceph/conf/ceph.crt
>> set mgr/restful/fa42/crt
>> 
>> The exact same certificate is accepted by the mgr.
>> 
>> 
>>> Le 3 mai 2021 à 23:03, Fabrice Bacchella 
>> a écrit :
>>> 
>>> Once activated the dashboard, I try to import certificates, but it fails:
>>> 
>>> $ ceph dashboard set-ssl-certificate-key -i /data/ceph/conf/ceph.key
>>> Error EINVAL: Traceback (most recent call last):
>>> File "/usr/share/ceph/mgr/mgr_module.py", line 1337, in _handle_command
>>>   return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
>>> File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
>>>   return self.func(mgr, **kwargs)
>>> File "/usr/share/ceph/mgr/dashboard/module.py", line 385, in
>> set_ssl_certificate_key
>>>   self.set_store('key', inbuf.decode())
>>> AttributeError: 'str' object has no attribute 'decode'
>>> 
>>> $ ceph dashboard set-ssl-certificate  -i /data/ceph/conf/ceph.crt
>>> Error EINVAL: Traceback (most recent call last):
>>> File "/usr/share/ceph/mgr/mgr_module.py", line 1337, in _handle_command
>>>   return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
>>> File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
>>>   return self.func(mgr, **kwargs)
>>> File "/usr/share/ceph/mgr/dashboard/module.py", line 372, in
>> set_ssl_certificate
>>>   self.set_store('crt', inbuf.decode())
>>> AttributeError: 'str' object has no attribute 'decode'
>>> 
>>> 
>>> They are both PEM encoded files:
>>> file /data/ceph/conf/ceph.key /data/ceph/conf/ceph.crt
>>> /data/ceph/conf/ceph.key: PEM RSA private key
>>> /data/ceph/conf/ceph.crt: PEM certificate
>>> 
>>> What format does this command expect ?
>>> 
>>> That error happens on Centos 8.3.2011 with ceph-mgr-16.2.1-0.el8.x86_64,
>> downloaded directly from ceph.
>>> ___
>>> ceph-users mailing list -- ceph-users@ceph.io
>>> To unsubscribe send an email to ceph-users-le...@ceph.io
>> 
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io
>> 
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Certificat format for the SSL dashboard

[Fabrice Bacchella]

And worst:

$ ceph config-key set mgr/restful/fa42/crt -i /data/ceph/conf/ceph.crt 
set mgr/restful/fa42/crt

The exact same certificate is accepted by the mgr.


> Le 3 mai 2021 à 23:03, Fabrice Bacchella  a 
> écrit :
> 
> Once activated the dashboard, I try to import certificates, but it fails:
> 
> $ ceph dashboard set-ssl-certificate-key -i /data/ceph/conf/ceph.key 
> Error EINVAL: Traceback (most recent call last):
>  File "/usr/share/ceph/mgr/mgr_module.py", line 1337, in _handle_command
>return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
>  File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
>return self.func(mgr, **kwargs)
>  File "/usr/share/ceph/mgr/dashboard/module.py", line 385, in 
> set_ssl_certificate_key
>self.set_store('key', inbuf.decode())
> AttributeError: 'str' object has no attribute 'decode'
> 
> $ ceph dashboard set-ssl-certificate  -i /data/ceph/conf/ceph.crt
> Error EINVAL: Traceback (most recent call last):
>  File "/usr/share/ceph/mgr/mgr_module.py", line 1337, in _handle_command
>return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
>  File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
>return self.func(mgr, **kwargs)
>  File "/usr/share/ceph/mgr/dashboard/module.py", line 372, in 
> set_ssl_certificate
>self.set_store('crt', inbuf.decode())
> AttributeError: 'str' object has no attribute 'decode'
> 
> 
> They are both PEM encoded files:
> file /data/ceph/conf/ceph.key /data/ceph/conf/ceph.crt
> /data/ceph/conf/ceph.key: PEM RSA private key
> /data/ceph/conf/ceph.crt: PEM certificate
> 
> What format does this command expect ?
> 
> That error happens on Centos 8.3.2011 with ceph-mgr-16.2.1-0.el8.x86_64, 
> downloaded directly from ceph.
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Certificat format for the SSL dashboard

[Fabrice Bacchella]

Once activated the dashboard, I try to import certificates, but it fails:

$ ceph dashboard set-ssl-certificate-key -i /data/ceph/conf/ceph.key 
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1337, in _handle_command
return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/module.py", line 385, in 
set_ssl_certificate_key
self.set_store('key', inbuf.decode())
AttributeError: 'str' object has no attribute 'decode'

$ ceph dashboard set-ssl-certificate  -i /data/ceph/conf/ceph.crt
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1337, in _handle_command
return CLICommand.COMMANDS[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/module.py", line 372, in 
set_ssl_certificate
self.set_store('crt', inbuf.decode())
AttributeError: 'str' object has no attribute 'decode'


They are both PEM encoded files:
file /data/ceph/conf/ceph.key /data/ceph/conf/ceph.crt
/data/ceph/conf/ceph.key: PEM RSA private key
/data/ceph/conf/ceph.crt: PEM certificate

What format does this command expect ?

That error happens on Centos 8.3.2011 with ceph-mgr-16.2.1-0.el8.x86_64, 
downloaded directly from ceph.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: How radosgw works ?

[Fabrice Bacchella]

I'm still trying to wrap my head around the radosgw when using fastcgi on a 
socket file.

First the documentation is very poor. The fcgi usage is documented in the man 
page, but the http frontends have a dedicated section. Why ?

Next, I don't understand if they are exclusive or not. I trying to define the 
configuration setting, a command line argument, with or without rgw_frontends 
defined.

Nothing changes. And I don't see any indications in the logs about even trying. 
strace does not provides any help. So whatever I do, radosgw never try to use 
socket fcgi. Why ?

I'm running on Centos 8, with rpm from ceph: ceph-radosgw-16.2.1-0.el8.x86_64 

> Le 24 avr. 2021 à 00:00, Konstantin Shalygin  a écrit :
> 
> 
> 
>> On 23 Apr 2021, at 21:05, Fabrice Bacchella > <mailto:fabrice.bacche...@orange.fr>> wrote:
>> 
>> Next about some naming, or binding IP. Where it's defined, and how ?
>> You have:
>> rgw_frontends = "beast ssl_endpoint=0.0.0.0:443 port=443 ..."
>> rgw_host =
>> rgw_port = 
>> rgw_dns_name =
>> 
>> That's a lot of redundancy, or contradictory informations. What is the 
>> purpose of each one ? What is the difference between
>> rgw_frontends = ".. port = ..."
>> and
>> rgw_port = 
>> 
>> ?
>> Or rgw_host and rgw_dns_name. What is the difference ?
>> 
>> The documentation provides no help at all:
>> rgw_dns_name
>> Description: The DNS name of the served domain. See also the hostnames 
>> setting within regions.
> 
> rgw_port is for fastcgi, default is unset
> rgw_frontends is civetweb or beast, for example you can configure civetweb 
> like this:
> 
> Listen on 443 securely with cert, redirect requests from 80 to 443:
> rgw frontends = civetweb port=0.0.0.0:80r+443s 
> ssl_certificate=/etc/pki/tls/private/example.com.pem 
> ssl_cipher_list=ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
> 
> rgw_dns_name is your endpoint. For example you run 10 redundant RGW's: for 
> each you need to define the same values of:
> rgw_zone = default
> rgw_zonegroup = default
> rgw_realm = default
> 
> For example your cert is FQDN: *.example.com <http://example.com/> and your 
> endpoint service is: swift.example.com <http://swift.example.com/>, that
> rgw_dns_name = swift.example.com <http://swift.example.com/>
> 
> Civetweb frontend is documented all possible options [1], for Beast should be 
> the same
> 
> 
> [1] http://civetweb.github.io/civetweb/UserManual.html 
> <http://civetweb.github.io/civetweb/UserManual.html>
> 
> Good luck,
> k

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: wrong socket path with ceph daemonperf

[Fabrice Bacchella]

I was thinking about reporting the bug at 
https://pad.ceph.com/p/Report_Documentation_Bugs 
<https://pad.ceph.com/p/Report_Documentation_Bugs>.

But what is thing thing ? What am I expected to do with that ? Just add a 
comment at the end ? But is this document read, used ? That's not a bugtrack 
but a shared document.

> Le 24 avr. 2021 à 08:50, Fabrice Bacchella  a 
> écrit :
> 
> 
> 
>> Le 23 avr. 2021 à 23:37, Konstantin Shalygin  a écrit :
>> 
>> Ceph project is drop cluster changing name may years ago. Is more client 
>> feature, for example
>> for OpenStack API's --cluster=${name} is a /etc/ceph/${name}.conf or other 
>> custom software for
>> multi cluster access from one point.
>> 
>> So it's unmaintained feature.
>> 
> OK it's explain here 
> https://docs.ceph.com/en/pacific/rados/configuration/common/#running-multiple-clusters-deprecated:
>  
> 
> But at https://docs.ceph.com/en/pacific/install/manual-deployment/:
> Cluster Name: Ceph clusters have a cluster name, which is a simple string 
> without spaces. The default cluster name is ceph, but you may specify a 
> different cluster name. Overriding the default cluster name is especially 
> useful when you are working with multiple clusters and you need to clearly 
> understand which cluster your are working with.
> 
> For example, when you run multiple clusters in a multisite configuration 
> <https://docs.ceph.com/en/pacific/radosgw/multisite/#multisite>, the cluster 
> name (e.g., us-west, us-east) identifies the cluster for the current CLI 
> session. Note: To identify the cluster name on the command line interface, 
> specify the Ceph configuration file with the cluster name (e.g., ceph.conf, 
> us-west.conf, us-east.conf, etc.). Also see CLI usage (ceph --cluster 
> {cluster-name}).
> 
> And many other places. So that paragraph is a documentation bug.
> 
>> 
>> 
>> k
>> 
>>> On 22 Apr 2021, at 23:13, Fabrice Bacchella >> <mailto:fabrice.bacche...@orange.fr>> wrote:
>>> 
>>> I doing a custom, manual installation of ceph. I changed the cluster name 
>>> from the default one to 'ngceph'.
>> 
> 
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: wrong socket path with ceph daemonperf

[Fabrice Bacchella]

> Le 23 avr. 2021 à 23:37, Konstantin Shalygin  a écrit :
> 
> Ceph project is drop cluster changing name may years ago. Is more client 
> feature, for example
> for OpenStack API's --cluster=${name} is a /etc/ceph/${name}.conf or other 
> custom software for
> multi cluster access from one point.
> 
> So it's unmaintained feature.
> 
OK it's explain here 
https://docs.ceph.com/en/pacific/rados/configuration/common/#running-multiple-clusters-deprecated:
 

But at https://docs.ceph.com/en/pacific/install/manual-deployment/:
Cluster Name: Ceph clusters have a cluster name, which is a simple string 
without spaces. The default cluster name is ceph, but you may specify a 
different cluster name. Overriding the default cluster name is especially 
useful when you are working with multiple clusters and you need to clearly 
understand which cluster your are working with.

For example, when you run multiple clusters in a multisite configuration 
<https://docs.ceph.com/en/pacific/radosgw/multisite/#multisite>, the cluster 
name (e.g., us-west, us-east) identifies the cluster for the current CLI 
session. Note: To identify the cluster name on the command line interface, 
specify the Ceph configuration file with the cluster name (e.g., ceph.conf, 
us-west.conf, us-east.conf, etc.). Also see CLI usage (ceph --cluster 
{cluster-name}).

And many other places. So that paragraph is a documentation bug.

> 
> 
> k
> 
>> On 22 Apr 2021, at 23:13, Fabrice Bacchella > <mailto:fabrice.bacche...@orange.fr>> wrote:
>> 
>> I doing a custom, manual installation of ceph. I changed the cluster name 
>> from the default one to 'ngceph'.
> 

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] How radosgw works ?

[Fabrice Bacchella]

I'm trying to understand what and where radosgw listen ?

There is a lot of contradictory or redundant informations about that.

First about the contradictory informations for the socket.

At https://docs.ceph.com/en/pacific/radosgw/config-ref/ 
, it says 
rgw_socket_path, but at https://docs.ceph.com/en/pacific/man/8/radosgw/ 
 is says 'rgw socket path'
That problem is quite common in the ceph documentation. Are both value accepted 
?

Next about some naming, or binding IP. Where it's defined, and how ?
You have:
rgw_frontends = "beast ssl_endpoint=0.0.0.0:443 port=443 ..."
rgw_host =
rgw_port = 
rgw_dns_name =

That's a lot of redundancy, or contradictory informations. What is the purpose 
of each one ? What is the difference between
rgw_frontends = ".. port = ..."
and
rgw_port = 

?
Or rgw_host and rgw_dns_name. What is the difference ?

The documentation provides no help at all:
rgw_dns_name
Description: The DNS name of the served domain. See also the hostnames setting 
within regions.

The description says nothing new, it just repeat the field name.

Is one of them used by the manager for communication ? I already had the 
problem for the entry in the certificate used by the frontend, it used an IP 
coming from nowhere.
If a fcgi is used, how the manager find the endpoint ?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] certificates between msg and radosgw

[Fabrice Bacchella]

I've just activated certificates on my radosgw, and the manager is unable to 
talk to it.

On the manager logs, I see:

2021-04-23T15:05:03.917+0200 7f4109309700  0 [dashboard ERROR 
urllib3.connection] Certificate did not match expected hostname: XXX 
Certificate: {...)}
2021-04-23T15:05:03.918+0200 7f4109309700  0 [dashboard ERROR exception] 
Internal Server Error
Traceback (most recent call last):
  File "/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in 
urlopen
chunked=chunked)
  File "/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in 
_make_request
self._validate_conn(conn)
  File "/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in 
_validate_conn
conn.connect()
  File "/lib/python3.6/site-packages/urllib3/connection.py", line 364, in 
connect
_match_hostname(cert, self.assert_hostname or server_hostname)
  File "/lib/python3.6/site-packages/urllib3/connection.py", line 374, in 
_match_hostname
match_hostname(cert, asserted_hostname)
  File "/lib64/python3.6/ssl.py", line 287, in match_hostname
% (hostname, ', '.join(map(repr, dnsnames
ssl.CertificateError: hostname 'SOMEIP' doesn't match either of 'XXX'

So the manager is communicating to the gateway using the IP. So I configured 
the names in ceph.conf:

[client.radosgw.ID]
  rgw_data = /data/ceph/data/radosgw/$cluster.ID
  log_file = /data/ceph/logs/$cluster-radosgw.ID.log
  rgw_frontends = "beast ssl_endpoint=0.0.0.0:443 
ssl_certificate=/data/ceph/conf/ceph.crt 
ssl_private_key=/data/ceph/conf/ceph.key"
  rgw_dns_name = XXX

But it's seems that the line rgw_dns_name is not used by the manager. Where 
does it resolve the name to use for the connection ?
 

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: wrong socket path with ceph daemonperf

[Fabrice Bacchella]

> Le 22 avr. 2021 à 22:13, Fabrice Bacchella  a 
> écrit :
> 
> I doing a custom, manual installation of ceph. I changed the cluster name 
> from the default one to 'ngceph'.
> 
> I changed the installation path to /data/ceph.
> 
> A lot of thing correctly resolves the settings. But one command fails:
> CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph daemonperf 
> client.radosgw.fa43 list
> ...
> Traceback (most recent call last):
>  File "/usr/lib/python3.6/site-packages/ceph_daemon.py", line 73, in 
> admin_socket
>b'{"prefix": "get_command_descriptions"}')
>  File "/usr/lib/python3.6/site-packages/ceph_daemon.py", line 49, in do_sockio
>sock.connect(path)
> FileNotFoundError: [Errno 2] No such file or directory
> ...
> 
> And indeed, after a few check, I found the culprit:
> CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph strace -f -e trace=file 
> ceph daemonperf client.radosgw.fa43 list
> ...
> [pid  3431] execve("/bin/ceph-conf", ["ceph-conf", "--name", 
> "client.radosgw.fa43", "--show-config-value", "admin_socket"], 0x5651c6e1cb30 
> /* 31 vars */) = 0
> 
> So I tried this command:
> 
> 
> CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph-conf --name 
> client.radosgw.fa43 --show-config-value admin_socket
> /var/run/ceph/ceph-client.radosgw.fa43.asok
> 

The command is wrong, but trying the good command doesn't change the result:

CEPH_CONF=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph-conf --name 
client.radosgw.fa43 --show-config-value admin_socket
/var/run/ceph/ceph-client.radosgw.fa43.asok

> But the real path for the socket is:
> 
> ls -l /var/run/ceph/ngceph-client.radosgw.fa43.2521.94787303273792.asok 
> srwxr-xr-x 1 ceph ceph 0 Apr 22 21:47 
> /var/run/ceph/ngceph-client.radosgw.fa43.2521.94787303273792.asok
> 
> It does not use the cluster name, and I wonder what is this 
> 2521.94787303273792 coming from. Would it fails even with a standard cluster 
> name ?
> 
> The version I'm using is a rpm build from ceph:
> $ rpm -qf /bin/ceph-conf
> ceph-common-16.2.1-0.el8.x86_64
> 
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] /ceph-osd-prestart.sh does use the configuration

[Fabrice Bacchella]

I'm using a rpm build directly from ceph, ceph-osd-16.2.1-0.el8.x86_64

I tweaked the configuration with a custom cluster name and custom installation 
dir

In the unit, there is the configuration:
ExecStartPre=/usr/libexec/ceph/ceph-osd-prestart.sh --cluster ${CLUSTER} --id %i

And this script check the osd path with an hard coded value:
data="/var/lib/ceph/osd/${cluster:-ceph}-$id"

It doesn't use the key from the configuration file. But using ceph-conf would 
have solve the problem:

$ CEPH_CONF=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph-conf --name osd.1 
--show-config-value osd_data
/data/ceph/data/osd/ceph.1

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] wrong socket path with ceph daemonperf

[Fabrice Bacchella]

I doing a custom, manual installation of ceph. I changed the cluster name from 
the default one to 'ngceph'.

I changed the installation path to /data/ceph.

A lot of thing correctly resolves the settings. But one command fails:
CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph daemonperf 
client.radosgw.fa43 list
...
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ceph_daemon.py", line 73, in 
admin_socket
b'{"prefix": "get_command_descriptions"}')
  File "/usr/lib/python3.6/site-packages/ceph_daemon.py", line 49, in do_sockio
sock.connect(path)
FileNotFoundError: [Errno 2] No such file or directory
...

And indeed, after a few check, I found the culprit:
CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph strace -f -e trace=file ceph 
daemonperf client.radosgw.fa43 list
...
[pid  3431] execve("/bin/ceph-conf", ["ceph-conf", "--name", 
"client.radosgw.fa43", "--show-config-value", "admin_socket"], 0x5651c6e1cb30 
/* 31 vars */) = 0

So I tried this command:


CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph-conf --name 
client.radosgw.fa43 --show-config-value admin_socket
/var/run/ceph/ceph-client.radosgw.fa43.asok

But the real path for the socket is:

ls -l /var/run/ceph/ngceph-client.radosgw.fa43.2521.94787303273792.asok 
srwxr-xr-x 1 ceph ceph 0 Apr 22 21:47 
/var/run/ceph/ngceph-client.radosgw.fa43.2521.94787303273792.asok

It does not use the cluster name, and I wonder what is this 2521.94787303273792 
coming from. Would it fails even with a standard cluster name ?

The version I'm using is a rpm build from ceph:
$ rpm -qf /bin/ceph-conf
ceph-common-16.2.1-0.el8.x86_64

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Configuring an S3 gateway

[Fabrice Bacchella]

> Le 22 avr. 2021 à 20:40, Casey Bodley  a écrit :
> 
> On Thu, Apr 22, 2021 at 2:26 PM Fabrice Bacchella
> mailto:fabrice.bacche...@orange.fr>> wrote:
>> 
>> I'm trying to configure an S3 gateway with pacific and can't wrap my mind 
>> around.
>> 
>> In the configuration file, my configuration is:
>> 
>> [client.radosgw.fa41]
>>  rgw_data = /data/ceph/data/radosgw/$cluster.$id
>>  log_file = /data/ceph/logs/$cluster-radosgw.$id.log
>>  rgw_frontends = "beast ssl_endpoint=0.0.0.0:443 
>> ssl_certificate=/data/ceph/conf/ceph.crt 
>> ssl_private_key=/data/ceph/conf/ceph.key"
>> 
>> and radosgw ignore it:
>> 
>> # /usr/bin/radosgw -d --cluster ngceph --name client.fa41 --setuser ceph 
>> --setgroup ceph
> 
> the --name should match the ceph.conf section. does '--name
> client.radosgw.fa41' work?

Indeed it works much better now, thanks.

But I took the argument from the unit:
ExecStart=/usr/bin/radosgw -f --cluster ${CLUSTER} --name client.%i --setuser 
ceph --setgroup ceph

So to works, the unit to start should be ceph-radosgw@radosgw.fa41.service. 
That's would be a little strange.

And in the dashboard, it's shown as with an ID of radosgw.fa41 in the 
ObjectGatway screen. Isn't that some kind of inconcistency about what is the ID 
for a client ?

Meanwhile, neither 
/usr/bin/radosgw -d --cluster ngceph -i radosgw.fa41 --setuser ceph --setgroup 
ceph
or
/usr/bin/radosgw -d --cluster ngceph -i fa41 --setuser ceph --setgroup ceph

 
I've installed ceph-radosgw-16.2.1-0.el8.x86_64 directly from the ceph 
repositories.


> 
>> 2021-04-22T20:19:44.362+0200 7fdf51416480  0 deferred set uid:gid to 167:167 
>> (ceph:ceph)
>> 2021-04-22T20:19:44.363+0200 7fdf51416480  0 ceph version 16.2.1 
>> (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable), process 
>> radosgw, pid 9780
>> 2021-04-22T20:19:44.363+0200 7fdf51416480  0 framework: beast
>> 2021-04-22T20:19:44.363+0200 7fdf51416480  0 framework conf key: port, val: 
>> 7480
>> 2021-04-22T20:19:44.363+0200 7fdf51416480  1 radosgw_Main not setting numa 
>> affinity
>> 2021-04-22T20:19:45.585+0200 7fdf51416480  0 framework: beast
>> 2021-04-22T20:19:45.586+0200 7fdf51416480  0 framework conf key: 
>> ssl_certificate, val: config://rgw/cert/$realm/$zone.crt
>> 2021-04-22T20:19:45.586+0200 7fdf51416480  0 framework conf key: 
>> ssl_private_key, val: config://rgw/cert/$realm/$zone.key
>> 2021-04-22T20:19:45.586+0200 7fdf51416480  0 starting handler: beast
>> 2021-04-22T20:19:45.592+0200 7fdf51416480  0 WARNING: cannot open socket for 
>> endpoint=[::]:7480, Address family not supported by protocol
>> 2021-04-22T20:19:45.627+0200 7fdf51416480  0 set uid:gid to 167:167 
>> (ceph:ceph)
>> 2021-04-22T20:19:45.811+0200 7fdf51416480  1 mgrc service_daemon_register 
>> rgw.134130 metadata {arch=x86_64,ceph_release=pacific,ceph_version=ceph 
>> version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific 
>> (stable),ceph_version_short=16.2.1,cpu=Intel Core Processor (Haswell, no 
>> TSX),distro=centos,distro_description=CentOS Linux 
>> 8,distro_version=8,frontend_config#0=beast 
>> port=7480,frontend_type#0=beast,hostname=fa41,id=fa41,kernel_description=#1 
>> SMP Thu Apr 8 19:01:30 UTC 
>> 2021,kernel_version=4.18.0-240.22.1.el8_3.x86_64,mem_swap_kb=0,mem_total_kb=16211232,num_handles=1,os=Linux,pid=9780,zone_id=2dc75a54-8c59-42bc-98a8-35542fdc4e52,zone_name=default,zonegroup_id=d11b8d14-7608-4b1d-a548-09b5dd813a7a,zonegroup_name=default}
>> 
>> I don't get what I'm missing. Is there any typo in the configuration that 
>> I'm missing ?
>> 
>> I've verified using strace, and it reads the expected configuration file.
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Configuring an S3 gateway

[Fabrice Bacchella]

I'm trying to configure an S3 gateway with pacific and can't wrap my mind 
around.

In the configuration file, my configuration is:

[client.radosgw.fa41]
  rgw_data = /data/ceph/data/radosgw/$cluster.$id
  log_file = /data/ceph/logs/$cluster-radosgw.$id.log
  rgw_frontends = "beast ssl_endpoint=0.0.0.0:443 
ssl_certificate=/data/ceph/conf/ceph.crt 
ssl_private_key=/data/ceph/conf/ceph.key"

and radosgw ignore it:

# /usr/bin/radosgw -d --cluster ngceph --name client.fa41 --setuser ceph 
--setgroup ceph 
2021-04-22T20:19:44.362+0200 7fdf51416480  0 deferred set uid:gid to 167:167 
(ceph:ceph)
2021-04-22T20:19:44.363+0200 7fdf51416480  0 ceph version 16.2.1 
(afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable), process radosgw, 
pid 9780
2021-04-22T20:19:44.363+0200 7fdf51416480  0 framework: beast
2021-04-22T20:19:44.363+0200 7fdf51416480  0 framework conf key: port, val: 7480
2021-04-22T20:19:44.363+0200 7fdf51416480  1 radosgw_Main not setting numa 
affinity
2021-04-22T20:19:45.585+0200 7fdf51416480  0 framework: beast
2021-04-22T20:19:45.586+0200 7fdf51416480  0 framework conf key: 
ssl_certificate, val: config://rgw/cert/$realm/$zone.crt
2021-04-22T20:19:45.586+0200 7fdf51416480  0 framework conf key: 
ssl_private_key, val: config://rgw/cert/$realm/$zone.key
2021-04-22T20:19:45.586+0200 7fdf51416480  0 starting handler: beast
2021-04-22T20:19:45.592+0200 7fdf51416480  0 WARNING: cannot open socket for 
endpoint=[::]:7480, Address family not supported by protocol
2021-04-22T20:19:45.627+0200 7fdf51416480  0 set uid:gid to 167:167 (ceph:ceph)
2021-04-22T20:19:45.811+0200 7fdf51416480  1 mgrc service_daemon_register 
rgw.134130 metadata {arch=x86_64,ceph_release=pacific,ceph_version=ceph version 
16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific 
(stable),ceph_version_short=16.2.1,cpu=Intel Core Processor (Haswell, no 
TSX),distro=centos,distro_description=CentOS Linux 
8,distro_version=8,frontend_config#0=beast 
port=7480,frontend_type#0=beast,hostname=fa41,id=fa41,kernel_description=#1 SMP 
Thu Apr 8 19:01:30 UTC 
2021,kernel_version=4.18.0-240.22.1.el8_3.x86_64,mem_swap_kb=0,mem_total_kb=16211232,num_handles=1,os=Linux,pid=9780,zone_id=2dc75a54-8c59-42bc-98a8-35542fdc4e52,zone_name=default,zonegroup_id=d11b8d14-7608-4b1d-a548-09b5dd813a7a,zonegroup_name=default}

I don't get what I'm missing. Is there any typo in the configuration that I'm 
missing ?

I've verified using strace, and it reads the expected configuration file.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io