[ceph-users] Re: Do people still use LevelDBStore?
I think it's a great idea to remove it. - Ken On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson wrote: > > Good day, > > Some time ago, the LevelDB maintainers turned -fno-rtti on in their > build. As we don't use -fno-rtti, building LevelDBStore > against newer LevelDB packages can fail. > > This has made me wonder, are there still people who use LevelDBStore > and rely on it, or can we deprecate and/or remove it? > > ___ > Dev mailing list -- d...@ceph.io > To unsubscribe send an email to dev-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
+1 from a dev's perspective. we don't test leveldb, and we don't expect it to perform as well as rocksdb in ceph, so i don't see any value in keeping it the rados team put a ton of effort into converting existing clusters to rocksdb, so i would be very surprised if removing leveldb left any users stuck without an upgrade path On Wed, Oct 13, 2021 at 2:13 PM Ken Dreyer wrote: > > I think it's a great idea to remove it. > > - Ken > > On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson wrote: > > > > Good day, > > > > Some time ago, the LevelDB maintainers turned -fno-rtti on in their > > build. As we don't use -fno-rtti, building LevelDBStore > > against newer LevelDB packages can fail. > > > > This has made me wonder, are there still people who use LevelDBStore > > and rely on it, or can we deprecate and/or remove it? > > > > ___ > > Dev mailing list -- d...@ceph.io > > To unsubscribe send an email to dev-le...@ceph.io > > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
+1 from users perspective too but we should warn add a HEALTH_WARN if a cluster has leveldb monstore or filestore osds, so users know to convert before upgrading. -- dan On Wed, Oct 13, 2021 at 8:39 PM Casey Bodley wrote: > > +1 from a dev's perspective. we don't test leveldb, and we don't > expect it to perform as well as rocksdb in ceph, so i don't see any > value in keeping it > > the rados team put a ton of effort into converting existing clusters > to rocksdb, so i would be very surprised if removing leveldb left any > users stuck without an upgrade path > > On Wed, Oct 13, 2021 at 2:13 PM Ken Dreyer wrote: > > > > I think it's a great idea to remove it. > > > > - Ken > > > > On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson > > wrote: > > > > > > Good day, > > > > > > Some time ago, the LevelDB maintainers turned -fno-rtti on in their > > > build. As we don't use -fno-rtti, building LevelDBStore > > > against newer LevelDB packages can fail. > > > > > > This has made me wonder, are there still people who use LevelDBStore > > > and rely on it, or can we deprecate and/or remove it? > > > > > > ___ > > > Dev mailing list -- d...@ceph.io > > > To unsubscribe send an email to dev-le...@ceph.io > > > > > > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
+1 we convert all levelbd monstore to rocksdb on luminous k Sent from my iPhone > On 14 Oct 2021, at 10:42, Dan van der Ster wrote: > > +1 from users perspective too but we should warn add a HEALTH_WARN if > a cluster has leveldb monstore or filestore osds, so users know to > convert before upgrading. > > -- dan > >> On Wed, Oct 13, 2021 at 8:39 PM Casey Bodley wrote: >> >> +1 from a dev's perspective. we don't test leveldb, and we don't >> expect it to perform as well as rocksdb in ceph, so i don't see any >> value in keeping it >> >> the rados team put a ton of effort into converting existing clusters >> to rocksdb, so i would be very surprised if removing leveldb left any >> users stuck without an upgrade path >> >>> On Wed, Oct 13, 2021 at 2:13 PM Ken Dreyer wrote: >>> >>> I think it's a great idea to remove it. >>> >>> - Ken >>> >>> On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson >>> wrote: Good day, Some time ago, the LevelDB maintainers turned -fno-rtti on in their build. As we don't use -fno-rtti, building LevelDBStore against newer LevelDB packages can fail. This has made me wonder, are there still people who use LevelDBStore and rely on it, or can we deprecate and/or remove it? ___ Dev mailing list -- d...@ceph.io To unsubscribe send an email to dev-le...@ceph.io >>> >>> ___ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-le...@ceph.io >>> >> >> ___ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
This is a very basic question, but how to quickly check if you still run levelbd instead of rocksdb? Op 14-10-2021 om 16:55 schreef Konstantin Shalygin: +1 we convert all levelbd monstore to rocksdb on luminous k Sent from my iPhone On 14 Oct 2021, at 10:42, Dan van der Ster wrote: +1 from users perspective too but we should warn add a HEALTH_WARN if a cluster has leveldb monstore or filestore osds, so users know to convert before upgrading. -- dan On Wed, Oct 13, 2021 at 8:39 PM Casey Bodley wrote: +1 from a dev's perspective. we don't test leveldb, and we don't expect it to perform as well as rocksdb in ceph, so i don't see any value in keeping it the rados team put a ton of effort into converting existing clusters to rocksdb, so i would be very surprised if removing leveldb left any users stuck without an upgrade path On Wed, Oct 13, 2021 at 2:13 PM Ken Dreyer wrote: I think it's a great idea to remove it. - Ken On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson wrote: Good day, Some time ago, the LevelDB maintainers turned -fno-rtti on in their build. As we don't use -fno-rtti, building LevelDBStore against newer LevelDB packages can fail. This has made me wonder, are there still people who use LevelDBStore and rely on it, or can we deprecate and/or remove it? ___ Dev mailing list -- d...@ceph.io To unsubscribe send an email to dev-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
For a mon: # cat /var/lib/ceph/mon/ceph-xxx/kv_backend rocksdb For an OSD, look for leveldb/rocksdb messages in the log. -- dan On Fri, Oct 15, 2021 at 9:14 AM mj wrote: > > This is a very basic question, but how to quickly check if you still run > levelbd instead of rocksdb? > > Op 14-10-2021 om 16:55 schreef Konstantin Shalygin: > > +1 we convert all levelbd monstore to rocksdb on luminous > > > > > > k > > > > Sent from my iPhone > > > >> On 14 Oct 2021, at 10:42, Dan van der Ster wrote: > >> > >> +1 from users perspective too but we should warn add a HEALTH_WARN if > >> a cluster has leveldb monstore or filestore osds, so users know to > >> convert before upgrading. > >> > >> -- dan > >> > >>> On Wed, Oct 13, 2021 at 8:39 PM Casey Bodley wrote: > >>> > >>> +1 from a dev's perspective. we don't test leveldb, and we don't > >>> expect it to perform as well as rocksdb in ceph, so i don't see any > >>> value in keeping it > >>> > >>> the rados team put a ton of effort into converting existing clusters > >>> to rocksdb, so i would be very surprised if removing leveldb left any > >>> users stuck without an upgrade path > >>> > On Wed, Oct 13, 2021 at 2:13 PM Ken Dreyer wrote: > > I think it's a great idea to remove it. > > - Ken > > On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson > wrote: > > > > Good day, > > > > Some time ago, the LevelDB maintainers turned -fno-rtti on in their > > build. As we don't use -fno-rtti, building LevelDBStore > > against newer LevelDB packages can fail. > > > > This has made me wonder, are there still people who use LevelDBStore > > and rely on it, or can we deprecate and/or remove it? > > > > ___ > > Dev mailing list -- d...@ceph.io > > To unsubscribe send an email to dev-le...@ceph.io > > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > > >>> > >>> ___ > >>> ceph-users mailing list -- ceph-users@ceph.io > >>> To unsubscribe send an email to ceph-users-le...@ceph.io > >> ___ > >> ceph-users mailing list -- ceph-users@ceph.io > >> To unsubscribe send an email to ceph-users-le...@ceph.io > > > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
Hi, So it seems we're still on leveldb. We're using ceph through our proxmox cluster, and we're on 14.2.22. We know we have some upgrading to do, and we expect that the excellent pve upgrade wiki will (as usual) inform us how to upgrade including leveldb to rocksdb. (even though at this moment, we don't find much info on converting from leveldb to rocksdb on the pve wiki) MJ Op 15-10-2021 om 09:30 schreef Dan van der Ster: For a mon: # cat /var/lib/ceph/mon/ceph-xxx/kv_backend rocksdb For an OSD, look for leveldb/rocksdb messages in the log. -- dan On Fri, Oct 15, 2021 at 9:14 AM mj wrote: This is a very basic question, but how to quickly check if you still run levelbd instead of rocksdb? Op 14-10-2021 om 16:55 schreef Konstantin Shalygin: +1 we convert all levelbd monstore to rocksdb on luminous k Sent from my iPhone On 14 Oct 2021, at 10:42, Dan van der Ster wrote: +1 from users perspective too but we should warn add a HEALTH_WARN if a cluster has leveldb monstore or filestore osds, so users know to convert before upgrading. -- dan On Wed, Oct 13, 2021 at 8:39 PM Casey Bodley wrote: +1 from a dev's perspective. we don't test leveldb, and we don't expect it to perform as well as rocksdb in ceph, so i don't see any value in keeping it the rados team put a ton of effort into converting existing clusters to rocksdb, so i would be very surprised if removing leveldb left any users stuck without an upgrade path On Wed, Oct 13, 2021 at 2:13 PM Ken Dreyer wrote: I think it's a great idea to remove it. - Ken On Wed, Oct 13, 2021 at 12:52 PM Adam C. Emerson wrote: Good day, Some time ago, the LevelDB maintainers turned -fno-rtti on in their build. As we don't use -fno-rtti, building LevelDBStore against newer LevelDB packages can fail. This has made me wonder, are there still people who use LevelDBStore and rely on it, or can we deprecate and/or remove it? ___ Dev mailing list -- d...@ceph.io To unsubscribe send an email to dev-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
> > For an OSD, look for leveldb/rocksdb messages in the log. > And what if nothing is in these logs? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
> > For an OSD, look for leveldb/rocksdb messages in the log. > > And what if nothing is in these logs? I think you can race over all content in the OSD dir, which incurs lots more IO but also works: cd /var/lib/ceph/osd/ceph-123 # grep -rsil leveldb current/meta current/meta/osd\usuperblock__0_23C2FCDE__none or # grep -rsil leveldb current/omap/ current/omap/MANIFEST-1524661 -- May the most significant bit of your life be positive. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
On Fri, Oct 15, 2021 at 10:55 AM Marc wrote: > > > > > For an OSD, look for leveldb/rocksdb messages in the log. > > > > And what if nothing is in these logs? Bluestore will be rocks (100% of the time IIUC). FileStore may be level -- check `ceph daemon osd.X perf dump` and look for a rocksdb section. -- dan ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
Hi!
Op 15-10-2021 om 11:03 schreef Dan van der Ster:
Bluestore will be rocks (100% of the time IIUC).
FileStore may be level -- check `ceph daemon osd.X perf dump` and look
for a rocksdb section.
So, we *are* bluestore, but your command returns leveldb:
root@pve1:~# cat /var/lib/ceph/mon/ceph-0/kv_backend
leveldb
At the same time, this returns rockdb:
root@pve1:~# ceph daemon osd.1 perf dump | grep rock
"rocksdb": {
"rocksdb_write_wal_time": {
"rocksdb_write_memtable_time": {
"rocksdb_write_delay_time": {
"rocksdb_write_pre_and_post_time": {
Does it make sense?
MJ
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
On Fri, Oct 15, 2021 at 1:38 PM mj wrote:
>
> Hi!
>
> Op 15-10-2021 om 11:03 schreef Dan van der Ster:
> > Bluestore will be rocks (100% of the time IIUC).
> >
> > FileStore may be level -- check `ceph daemon osd.X perf dump` and look
> > for a rocksdb section.
>
> So, we *are* bluestore, but your command returns leveldb:
>
> > root@pve1:~# cat /var/lib/ceph/mon/ceph-0/kv_backend
> > leveldb
>
This means that that mon is using leveldb for it's monstore. (That's
where the central cluster maps are stored, for example).
> At the same time, this returns rockdb:
>
> > root@pve1:~# ceph daemon osd.1 perf dump | grep rock
> > "rocksdb": {
> > "rocksdb_write_wal_time": {
> > "rocksdb_write_memtable_time": {
> > "rocksdb_write_delay_time": {
> > "rocksdb_write_pre_and_post_time": {
>
> Does it make sense?
And this OSD is bluestore/rocksdb. Yes it all makes sense.
You'll need to redeploy the mons so that their new monstore is rocksdb.
The procedure depends on your orchestration. In our case it will be
something like
systemctl stop ceph-mon.target
cd /var/lib/ceph/mon
mv ceph-x ceph-x.bak
# run puppet
This is like restarting a mon with a new empty monstore, so it
synchronizes with the other mons to re-join the quorum.
Clearly you need to do this one at a time; and if anything goes wrong
with the recreation, just mv the mon store back into it's original
path and restart it.
Cheers, Dan
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
Den fre 15 okt. 2021 kl 13:38 skrev mj :
> Op 15-10-2021 om 11:03 schreef Dan van der Ster:
> > Bluestore will be rocks (100% of the time IIUC).
> >
> > FileStore may be level -- check `ceph daemon osd.X perf dump` and look
> > for a rocksdb section.
>
> So, we *are* bluestore, but your command returns leveldb:
> > root@pve1:~# cat /var/lib/ceph/mon/ceph-0/kv_backend
> > leveldb
That is the mon, OSDs are filestore or bluestore, and "bluestore is
always rocksdb" still holds true.
filestore OSDs may be leveldb or rocksdb.
> At the same time, this returns rockdb:
>
> > root@pve1:~# ceph daemon osd.1 perf dump | grep rock
> > "rocksdb": {
> > "rocksdb_write_wal_time": {
> > "rocksdb_write_memtable_time": {
> > "rocksdb_write_delay_time": {
> > "rocksdb_write_pre_and_post_time": {
>
> Does it make sense?
Yes, except you asked two different daemons.
--
May the most significant bit of your life be positive.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Do people still use LevelDBStore?
Hi Janne and Dan,
Thanks for sharing your insights!
MJ
Op 15-10-2021 om 16:07 schreef Janne Johansson:
Den fre 15 okt. 2021 kl 13:38 skrev mj :
Op 15-10-2021 om 11:03 schreef Dan van der Ster:
Bluestore will be rocks (100% of the time IIUC).
FileStore may be level -- check `ceph daemon osd.X perf dump` and look
for a rocksdb section.
So, we *are* bluestore, but your command returns leveldb:
root@pve1:~# cat /var/lib/ceph/mon/ceph-0/kv_backend
leveldb
That is the mon, OSDs are filestore or bluestore, and "bluestore is
always rocksdb" still holds true.
filestore OSDs may be leveldb or rocksdb.
At the same time, this returns rockdb:
root@pve1:~# ceph daemon osd.1 perf dump | grep rock
"rocksdb": {
"rocksdb_write_wal_time": {
"rocksdb_write_memtable_time": {
"rocksdb_write_delay_time": {
"rocksdb_write_pre_and_post_time": {
Does it make sense?
Yes, except you asked two different daemons.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
