Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
On 02/18/2018 09:20 AM, Sergey Malinin wrote: All I got with script replacement is the following: Oooh. Try something earlier, without this function https://github.com/ceph/ceph/blob/1adf4325455ccdbf03da63046c510bb041ac8520/src/pybind/mgr/prometheus/module.py To be clear: prometheus module was never work out of box at 12.2.2 k ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
All I got with script replacement is the following: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond response.body = self.handler() File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 217, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 61, in __call__ return self.callable(*self.args, **self.kwargs) File "/usr/lib/ceph/mgr/prometheus/module.py", line 409, in metrics if global_instance().have_mon_connection(): AttributeError: 'Module' object has no attribute 'have_mon_connection' I'm not familiar with Python, so any advise is much appreciated. On Friday, February 16, 2018 at 12:10, Konstantin Shalygin wrote: > > i just try to get the prometheus plugin up and runing > > > > Use module from master. > > From this commit should work with 12.2.2, just wget it and replace > stock module. > > https://github.com/ceph/ceph/blob/d431de74def1b8889ad568ab99436362833d063e/src/pybind/mgr/prometheus/module.py > > > > > k > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com (mailto:ceph-users@lists.ceph.com) > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
hi,
wihle i added the "class" to all my OSD's the ceph-mgr crashed :-( but
the prometheus plugin works now
for i in {1..9}; do ceph osd crush set-device-class hdd osd.$i; done
Thanks,
Ansgar
2018-02-16 10:12 GMT+01:00 Jan Fajerski :
> On Fri, Feb 16, 2018 at 09:27:08AM +0100, Ansgar Jazdzewski wrote:
>>
>> Hi Folks,
>>
>> i just try to get the prometheus plugin up and runing but as soon as i
>> browse /metrics i got:
>>
>> 500 Internal Server Error
>> The server encountered an unexpected condition which prevented it from
>> fulfilling the request.
>>
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line
>> 670, in respond
>>response.body = self.handler()
>> File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py",
>> line 217, in __call__
>>self.body = self.oldhandler(*args, **kwargs)
>> File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py",
>> line 61, in __call__
>>return self.callable(*self.args, **self.kwargs)
>> File "/usr/lib/ceph/mgr/prometheus/module.py", line 386, in metrics
>>metrics = global_instance().collect()
>> File "/usr/lib/ceph/mgr/prometheus/module.py", line 323, in collect
>>self.get_metadata_and_osd_status()
>> File "/usr/lib/ceph/mgr/prometheus/module.py", line 283, in
>> get_metadata_and_osd_status
>>dev_class['class'],
>> KeyError: 'class'
>
> This error is part of the osd metadata metric. Which version of Ceph are you
> running this with? Specifically the Crush Map of this cluster seems to not
> have the device class for each OSD yet.
>>
>>
>> I assume that i have to change the mkgr cephx kex? but iam not 100% sure
>>
>> mgr.mgr01
>> key: AQAqLIRasocnChAAbOIEMKVEWWHCbgVeEctwng==
>> caps: [mds] allow *
>> caps: [mon] allow profile mgr
>> caps: [osd] allow *
>>
>> thanks for your help,
>> Ansgar
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
> --
> Jan Fajerski
> Engineer Enterprise Storage
> SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> HRB 21284 (AG Nürnberg)
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
we upgraded the cluster from jewel to luminous
but the restart of the ceph osd service did not add the 'CLASS
(hdd/ssd)' on its own it it not exist so i had to add it on my own to
make it work.
should somhow mentiond in the upgrade process?
like:
for all osd make shure that the "class" it set in in the crushmap by
useing "ceph osd crush set-device-class hdd osd."
the versions on the stage cluster
# ceph versions
{
"mon": {
"ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba)
luminous (stable)": 3
},
"mgr": {
"ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba)
luminous (stable)": 1
},
"osd": {
"ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba)
luminous (stable)": 10
},
"mds": {
"ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba)
luminous (stable)": 2
},
"rgw": {
"ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba)
luminous (stable)": 3
},
"overall": {
"ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba)
luminous (stable)": 19
}
}
2018-02-16 12:09 GMT+01:00 John Spray :
> On Fri, Feb 16, 2018 at 8:27 AM, Ansgar Jazdzewski
> wrote:
>> Hi Folks,
>>
>> i just try to get the prometheus plugin up and runing but as soon as i
>> browse /metrics i got:
>>
>> 500 Internal Server Error
>> The server encountered an unexpected condition which prevented it from
>> fulfilling the request.
>>
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line
>> 670, in respond
>> response.body = self.handler()
>> File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py",
>> line 217, in __call__
>> self.body = self.oldhandler(*args, **kwargs)
>> File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py",
>> line 61, in __call__
>> return self.callable(*self.args, **self.kwargs)
>> File "/usr/lib/ceph/mgr/prometheus/module.py", line 386, in metrics
>> metrics = global_instance().collect()
>> File "/usr/lib/ceph/mgr/prometheus/module.py", line 323, in collect
>> self.get_metadata_and_osd_status()
>> File "/usr/lib/ceph/mgr/prometheus/module.py", line 283, in
>> get_metadata_and_osd_status
>> dev_class['class'],
>> KeyError: 'class'
>
> Looks a lot like you might be trying to run a >= luminous ceph-mgr
> with an older Ceph cluster?
>
> What versions are you running?
>
> John
>
>>
>> I assume that i have to change the mkgr cephx kex? but iam not 100% sure
>>
>> mgr.mgr01
>>key: AQAqLIRasocnChAAbOIEMKVEWWHCbgVeEctwng==
>>caps: [mds] allow *
>>caps: [mon] allow profile mgr
>>caps: [osd] allow *
>>
>> thanks for your help,
>> Ansgar
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
On Fri, Feb 16, 2018 at 8:27 AM, Ansgar Jazdzewski wrote: > Hi Folks, > > i just try to get the prometheus plugin up and runing but as soon as i > browse /metrics i got: > > 500 Internal Server Error > The server encountered an unexpected condition which prevented it from > fulfilling the request. > > Traceback (most recent call last): > File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line > 670, in respond > response.body = self.handler() > File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", > line 217, in __call__ > self.body = self.oldhandler(*args, **kwargs) > File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", > line 61, in __call__ > return self.callable(*self.args, **self.kwargs) > File "/usr/lib/ceph/mgr/prometheus/module.py", line 386, in metrics > metrics = global_instance().collect() > File "/usr/lib/ceph/mgr/prometheus/module.py", line 323, in collect > self.get_metadata_and_osd_status() > File "/usr/lib/ceph/mgr/prometheus/module.py", line 283, in > get_metadata_and_osd_status > dev_class['class'], > KeyError: 'class' Looks a lot like you might be trying to run a >= luminous ceph-mgr with an older Ceph cluster? What versions are you running? John > > I assume that i have to change the mkgr cephx kex? but iam not 100% sure > > mgr.mgr01 >key: AQAqLIRasocnChAAbOIEMKVEWWHCbgVeEctwng== >caps: [mds] allow * >caps: [mon] allow profile mgr >caps: [osd] allow * > > thanks for your help, > Ansgar > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
tanks i will have a look into it -- Ansgar 2018-02-16 10:10 GMT+01:00 Konstantin Shalygin : >> i just try to get the prometheus plugin up and runing > > > > Use module from master. > > From this commit should work with 12.2.2, just wget it and replace stock > module. > > https://github.com/ceph/ceph/blob/d431de74def1b8889ad568ab99436362833d063e/src/pybind/mgr/prometheus/module.py > > > > > k > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
On Fri, Feb 16, 2018 at 09:27:08AM +0100, Ansgar Jazdzewski wrote: Hi Folks, i just try to get the prometheus plugin up and runing but as soon as i browse /metrics i got: 500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request. Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond response.body = self.handler() File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 217, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 61, in __call__ return self.callable(*self.args, **self.kwargs) File "/usr/lib/ceph/mgr/prometheus/module.py", line 386, in metrics metrics = global_instance().collect() File "/usr/lib/ceph/mgr/prometheus/module.py", line 323, in collect self.get_metadata_and_osd_status() File "/usr/lib/ceph/mgr/prometheus/module.py", line 283, in get_metadata_and_osd_status dev_class['class'], KeyError: 'class' This error is part of the osd metadata metric. Which version of Ceph are you running this with? Specifically the Crush Map of this cluster seems to not have the device class for each OSD yet. I assume that i have to change the mkgr cephx kex? but iam not 100% sure mgr.mgr01 key: AQAqLIRasocnChAAbOIEMKVEWWHCbgVeEctwng== caps: [mds] allow * caps: [mon] allow profile mgr caps: [osd] allow * thanks for your help, Ansgar ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Jan Fajerski Engineer Enterprise Storage SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph-mgr Python error with prometheus plugin
i just try to get the prometheus plugin up and runing Use module from master. From this commit should work with 12.2.2, just wget it and replace stock module. https://github.com/ceph/ceph/blob/d431de74def1b8889ad568ab99436362833d063e/src/pybind/mgr/prometheus/module.py k ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Ceph-mgr Python error with prometheus plugin
Hi Folks, i just try to get the prometheus plugin up and runing but as soon as i browse /metrics i got: 500 Internal Server Error The server encountered an unexpected condition which prevented it from fulfilling the request. Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond response.body = self.handler() File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 217, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 61, in __call__ return self.callable(*self.args, **self.kwargs) File "/usr/lib/ceph/mgr/prometheus/module.py", line 386, in metrics metrics = global_instance().collect() File "/usr/lib/ceph/mgr/prometheus/module.py", line 323, in collect self.get_metadata_and_osd_status() File "/usr/lib/ceph/mgr/prometheus/module.py", line 283, in get_metadata_and_osd_status dev_class['class'], KeyError: 'class' I assume that i have to change the mkgr cephx kex? but iam not 100% sure mgr.mgr01 key: AQAqLIRasocnChAAbOIEMKVEWWHCbgVeEctwng== caps: [mds] allow * caps: [mon] allow profile mgr caps: [osd] allow * thanks for your help, Ansgar ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
