Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-04-30 Thread Roland0


mamema wrote: 
> have this in the logs today:
> 
> [21-04-29 09:51:04.8337] Slim::Networking::Repositories::__ANON__ (147)
> Failed to fetch https://www.nexus0.net/pub/sw/lmsdboptim/repo.xml:
> Connect timed out: Inappropriate ioctl for device
> [21-04-29 09:51:04.8340] Slim::Plugin::Extensions::Plugin::_noResponse
> (476) error fetching https://www.nexus0.net/pub/sw/lmsdboptim/repo.xml -
> Connect timed out: Inappropriate ioctl for device
> 

Can't reproduce, d/l works fine here, lots of successful accesses in the
web sever logs. Have you tried a different computer/client (browser,
wget, ...) ?



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | DB Optimizer | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-04-29 Thread mamema


have this in the logs today:

[21-04-29 09:51:04.8337] Slim::Networking::Repositories::__ANON__ (147)
Failed to fetch https://www.nexus0.net/pub/sw/lmsdboptim/repo.xml:
Connect timed out: Inappropriate ioctl for device
[21-04-29 09:51:04.8340] Slim::Plugin::Extensions::Plugin::_noResponse
(476) error fetching https://www.nexus0.net/pub/sw/lmsdboptim/repo.xml -
Connect timed out: Inappropriate ioctl for device

huh?



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-08 Thread Roland0


SpiderJon wrote: 
> I ran 
> 
> find /usr/share/squeezeboxserver -name SQLite.so  
> 
> which gave
> 
> 
/usr/share/squeezeboxserver/CPAN/arch/5.20/arm-linux-gnueabi-thread-multi-64int/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.20/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.26/aarch64-linux-thread-multi/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.26/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.22/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.28/aarch64-linux-thread-multi/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.24/aarch64-linux-thread-multi/auto/DBD/SQLite/SQLite.so
> /usr/share/squeezeboxserver/CPAN/arch/5.24/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
> 
> Not sure why so many are listed (?), but according to my LMS info page:
> 
> 
Perl Version: 5.28.1 -
> arm-linux-gnueabihf-thread-multi-64int
> 
The debian package contains binaries for various perl version/platform
combinations, not only the one you are actually using.
The relevant path for you is:
/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/

I've created a package you can download 'here'
(https://nexus0.net/pub/sw/misc/armv7l-linux-thread-multi-64int-5.28.tar.gz)

Replace your modules with the ones contained in the archive (i.e. only
DBD  DBI  DBI.pm  Image auto/DBD  auto/DBI  auto/Image)

Note:
- I'm not 100% sure these will work on an RPi (I've build them on an
Odroid 32bit Arm, so they -should- be compatible, but I can't actually
test it)
- First, make a backup (e.g. tar cf -
/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/|gzip
-9 > ~/lms_mods_arm-linux-gnueabihf-thread-multi-64int.tar.gz)
- worst case you can always re-install LMS from the debian package
- make sure you get all the paths right
- if something about the procedure is unclear, just ask



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | DB Optimizer | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread SpiderJon


Roland0 wrote: 
> Which perl version are you using?
> 
> Find your current module:
> > 
Code:

  >   > find /usr/share/squeezeboxserver -name SQLite.so
  > 

> > 
> and then run (substituting your path)
> > 
Code:

  >   > file 
/usr/share/squeezeboxserver/CPAN/arch/5.28/armv7l-linux-thread-multi/auto/DBD/SQLite/SQLite.so

> > 
> and post both results

find /usr/share/squeezeboxserver -name SQLite.so  gives


/usr/share/squeezeboxserver/CPAN/arch/5.20/arm-linux-gnueabi-thread-multi-64int/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.20/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.26/aarch64-linux-thread-multi/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.26/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.22/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.28/aarch64-linux-thread-multi/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.24/aarch64-linux-thread-multi/auto/DBD/SQLite/SQLite.so
/usr/share/squeezeboxserver/CPAN/arch/5.24/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so

Not sure why so many are listed (?), but according to my LMS info page:


Perl Version: 5.28.1 -
arm-linux-gnueabihf-thread-multi-64int

So I ran

file
/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so

which gives


/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/auto/DBD/SQLite/SQLite.so:
ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically
linked, stripped

Hope that's what you need.



Duet x 3, via various hi-fi. LMS on Raspberry Pi OS (Debian 10) on a
Pi4.

SpiderJon's Profile: http://forums.slimdevices.com/member.php?userid=21798
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread Roland0


I've just published a new version. There are quite a number of changes,
so if you notice something, feel free to provide feedback.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | DB Optimizer | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread mamema


mherger wrote: 
> > 
> Ah, no, not with the docker image. You'd have to build your own (based 
> on ours), or run LMS without docker or whatever.

No hat was not the intention of this test, so we're staying with sqlite
then.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread Michael Herger

there is no MySQL driver around.installing it? Hmm, its docker...
you are maintaining the docker image, sure for fun to test, but you
still sure it should work?


Ah, no, not with the docker image. You'd have to build your own (based 
on ours), or run LMS without docker or whatever.

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread mamema


mherger wrote: 
> > this doesn't seem to be the case with 8.2.
> [/color]
> 
> Double/triple check the connection string. Setting LMS up to use MySQL 
> might be fiddly. But it should definitely work to a certain degree. 
> Meaning there might be bugs around the SQL dialect being used or 
> whatever. But it should start up and scan. I know there are user doing
> this.

i found something:
setting dbtype to MySQL in server.prefs prevented LMS to revert back to
sqlite, but :

Perhaps the DBD::mysql perl module hasn't been fully installed,
or perhaps the capitalisation of 'mysql' isn't right.
Available drivers: DBM, ExampleP, File, Gofer, Proxy, SQLite, Sponge.
at /lms/CPAN/DBIx/Class/Storage/DBI.pm line 999.
[21-03-06 10:47:56.1810] Slim::Schema::Storage::throw_exception (121)
Backtrace

there is no MySQL driver around.installing it? Hmm, its docker...
you are maintaining the docker image, sure for fun to test, but you
still sure it should work?



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread Michael Herger

this doesn't seem to be the case with 8.2.


It certainly still should be possible.


I've created the database in mysql, created the suitable user (according
to server.prefs), granted rights to the database, changed dbsource entry
to mysql according to perl notation, i even flushed the cache dir.
LMS starts and a few secs later, the dbsource entry in server.prefs
reverts back to dbsource sqlite


Double/triple check the connection string. Setting LMS up to use MySQL 
might be fiddly. But it should definitely work to a certain degree. 
Meaning there might be bugs around the SQL dialect being used or 
whatever. But it should start up and scan. I know there are user doing this.

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-06 Thread mamema


mherger wrote: 
> > 
> But you should be able to use MySQL, if you wanted to give it a try. 
> Would be interesting if that would turn out to be faster. One advantage
> 
> certainly would be that it would use another core on today's multi-core
> 
> systems, whereas SQLite is running inside the single LMS thread.

this doesn't seem to be the case with 8.2.
I've created the database in mysql, created the suitable user (according
to server.prefs), granted rights to the database, changed dbsource entry
to mysql according to perl notation, i even flushed the cache dir.
LMS starts and a few secs later, the dbsource entry in server.prefs
reverts back to dbsource sqlite

So not test successful with mysql



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-05 Thread Roland0

SpiderJon wrote: 
> Yes, please. I'm running Debian 10 (Buster) on a Pi4 (armv7l-linux) — so
> 32bit Arm.

Which perl version are you using?

Find your current module:

Code:

find /usr/share/squeezeboxserver -name SQLite.so
  


and then run (substituting your path)

Code:

file 
/usr/share/squeezeboxserver/CPAN/arch/5.28/armv7l-linux-thread-multi/auto/DBD/SQLite/SQLite.so


and post both results



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | DB Optimizer | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-05 Thread Roland0


mherger wrote: 
> >> Mine is 440Mb - with around 25k artists although not 100% good tags
> and
> >> many obscure artists that probably have no image.
> > While my library only has ~5000 artists, this still doesn't account
> for
> > the DB size difference.
> 
> Haha... I only have a few hundred album artists!
> 
> What are your exact MAI settings (screenshot)?

(continued in the 'MAI thread'
(https://forums.slimdevices.com/showthread.php?99537-Announce-Music-amp-Artist-Information-plugin&p=1012948&viewfull=1#post1012948))



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | DB Optimizer | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-05 Thread SpiderJon

Roland0 wrote: 
> It's part of the perl module DBD::SQLite's source code, which
> unfortunately means you have to re-compile it.
> On the plus side, it only takes a few minutes and you get a shiny new
> SQLite version for free.
> 
> Or, if your LMS is running on 32bit Arm or 64bit x86, I could upload
> some binaries which may work.

Yes, please. I'm running Debian 10 (Buster) on a Pi4 (armv7l-linux) — so
32bit Arm.



Duet x 3, via various hi-fi. LMS on Raspberry Pi OS (Debian 10) on a
Pi4.

SpiderJon's Profile: http://forums.slimdevices.com/member.php?userid=21798
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-05 Thread Michael Herger

Mine is 440Mb - with around 25k artists although not 100% good tags and
many obscure artists that probably have no image.

While my library only has ~5000 artists, this still doesn't account for
the DB size difference.


Haha... I only have a few hundred album artists!

What are your exact MAI settings (screenshot)?
___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-05 Thread Michael Herger

i will, as i said, it's pure fun for me.
As you seems to be very knowlegeable with databases, do you have
suggestions to test the performance in a more "scientific" way, than
"seems faster"..?


You could write a script to run a bunch of requests: get a list of x 
albums, get the artwork for all these albums in a pre-cached size etc. 
Run the script against both/all configurations several times with server 
restarts in between to start cold.


Good benchmarking is a science in itself :-). Because you should also 
take into account that the OS might be caching, too etc.


___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


mamema wrote: 
> i will, as i said, it's pure fun for me.
> As you seems to be very knowlegeable with databases, do you have
> suggestions to test the performance in a more "scientific" way, than
> "seems faster"..?
> 
As luck would have it, 'there actually is a benchmark'
(https://nexus0.net/pub/sw/lmsvarscripts/#lms-benchmark) you could use.
Will need a bit of configuration for your setup.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread mamema


Roland0 wrote: 
> While my library only has ~5000 artists, this still doesn't account for
> the DB size difference.
> Do you have " Find artwork online during library scan" enabled (I
> don't)?
> 
> Whatever the reason, it seems to make sense to add imageproxy.db to the
> DB supported by the plugin. I'll look into this for the next update.

about 5000 artists also, imageproxy database 470 mb



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread mamema


Roland0 wrote: 
> 
> Nevertheless, I agree, if anyone bothers to invest time in actually
> benchmarking this, it'll interesting to the see the results.
> 

i will, as i said, it's pure fun for me.
As you seems to be very knowlegeable with databases, do you have
suggestions to test the performance in a more "scientific" way, than
"seems faster"..?
As i already have a mysql server running and LMS is dockerized, i can
duplicate the environment "file based, server based db" very easily.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


mherger wrote: 
> > I am (artists only), but my imageproxy.db is only 8MB (even though
> I've
> > checked " Download and pre-cache artist pictures from online
> sources"),
> > so I haven't bothered with it.
> > Do you have an idea what typical sizes would be?
> 
> Hmm... mine is about 770MB! Pre-caching actually should be using it. 
> That's odd.
Indeed.

There is some writing going on, though:

Code:


  $  ls -hl /var/lib/logitechmediaserver/cache/img*
  -rw-r--r-- 1 squeezeserver squeezeserver 8.0M Mar  4 15:25 
/var/lib/logitechmediaserver/cache/imgproxy.db
  -rw-r--r-- 1 squeezeserver squeezeserver  32K Mar  5 07:27 
/var/lib/logitechmediaserver/cache/imgproxy.db-shm
  -rw-r--r-- 1 squeezeserver squeezeserver 487K Mar  5 07:28 
/var/lib/logitechmediaserver/cache/imgproxy.db-wal
  



Any idea how to analyze this?



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


Paul Webster wrote: 
> Mine is 440Mb - with around 25k artists although not 100% good tags and
> many obscure artists that probably have no image.
While my library only has ~5000 artists, this still doesn't account for
the DB size difference.
Do you have " Find artwork online during library scan" enabled (I
don't)?

Whatever the reason, it seems to make sense to add imageproxy.db to the
DB supported by the plugin. I'll look into this for the next update.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Michael Herger

I am (artists only), but my imageproxy.db is only 8MB (even though I've
checked " Download and pre-cache artist pictures from online sources"),
so I haven't bothered with it.
Do you have an idea what typical sizes would be?


Hmm... mine is about 770MB! Pre-caching actually should be using it. 
That's odd.

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


mamema wrote: 
> i'm on of those guys willing to throw more RAM on the issue. I'm also
> willing to test (with you) how much more RAM is really needed to speed
> up the things. I have 32 GB which are more or less usable. But i also
> think, there must be a break even point where adding more RAM doesn't
> help anymore.
> 
For SQLite: Optimal database cache size is the file size of the database



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


Shozzer wrote: 
> II did a complete clear and rescan but funnily enough that seemed to
> take longer. I will have to keep an eye on that.
> 
I'm moderately confident that the plugin's DB settings do not adversely
affect performance (unless the system runs out of memory, but in this
case, the DB performance settings shouldn't have been "Maximum" in the
first place)
However, if this persists, the next step would be to determine during
which step of the re-scan the slowdown occurs,

> 
> There is one side effect I am experiencing. The Control Panel refuses to
> open. I restarted my Windows 10 PC and tried to open the panel. It
> opened straight away. I closed it and tried again a short while after
> and now it refuses to open. Any pointers on this please?
tbh, my first take would be that it's unrelated to the plugin. However,
this should be easy to check; De-activate the plugin, restart LMS, check
if it happens again (under the same circumstances).



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


SpiderJon wrote: 
> 
> Re: Re-enabling the sqlite_stat4 table in "Other methods to improve
> performance" on your 'LMS DB Optimizer info page'
> (https://www.nexus0.net/pub/sw/lmsdboptim/).
> Where is  Makefile.PL?  And is there anything else you need to do after
> uncommenting the line '-DSQLITE_ENABLE_STAT4'  to get things going?
> 

It's part of the perl module DBD::SQLite's source code, which
unfortunately means you have to re-compile it.
On the plus side, it only takes a few minutes and you get a shiny new
SQLite version for free.

Or, if your LMS is running on 32bit Arm or 64bit x86, I could upload
some binaries which may work.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Paul Webster


Roland0 wrote: 
> I am (artists only), but my imageproxy.db is only 8MB (even though I've
> checked " Download and pre-cache artist pictures from online sources"),
> so I haven't bothered with it.
> Do you have an idea what typical sizes would be?

Mine is 440Mb - with around 25k artists although not 100% good tags and
many obscure artists that probably have no image.



Paul Webster
http://dabdig.blogspot.com
author of \"now playing\" plugins covering radio france (fip etc), kcrw,
supla finland, abc australia, cbc/radio-canada and rte ireland

Paul Webster's Profile: http://forums.slimdevices.com/member.php?userid=105
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


mherger wrote: 
> 
> But you should be able to use MySQL, if you wanted to give it a try. 
> Would be interesting if that would turn out to be faster. One advantage
> 
> certainly would be that it would use another core on today's multi-core
> 
> systems,
> 
I'd speculate MySQL won't be faster.
There are a couple of methods to exploit multi-threading. Ignoring
processing several SQL queries at once (as LMS sends queries
sequentially) and more esoteric ones (data sharding etc.), this
basically leaves intra query parallelism, e.g.
- if a query has subqueries, processing those in parallel
- under certain (limited) conditions, certain ops during query
processing can be parallelized (e.g. higher level (table scans etc.,
usually requires explicit declaration), lower level (e.g. sorting
results))
MySQL will be better at this, but SQLite actually supports a (very)
limited form of the latter approach (iirc sorting results, maybe more?).
Multi-threading is, however, disabled by default. My plugin enables it
(2 threads), but atm I have no idea if it is ever used.
On the flip side, SQLite has an advantage in having siqnificantly less
overhead (no server etc.) and no intra-process communication.
And there is a real question if LMS even uses single queries worth
parallelizing (in terms of both complexity and data volume), as this
generates overhead of it's own (thus, SQLite has built-in heuristics to
decide how many threads it uses for a query).

Nevertheless, I agree, if anyone bothers to invest time in actually
benchmarking this, it'll interesting to the see the results.

mherger wrote: 
> 
> SQLite is running inside the single LMS thread.
> 
I don't think this prevents SQLite from launching additional threads if
it is so inclined.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


mherger wrote: 
> > I'm still fiddling with some of the parameters (cache sizes, mainly).
> > Currently, they should be fine for ~1-2GB free memory.
> 
> I'll have to play with this, in particular with the imageproxy.db, too:
> 
> MAI makes heavy use of it to cache artist artwork etc... Are you using
> MAI?
I am (artists only), but my imageproxy.db is only 8MB (even though I've
checked " Download and pre-cache artist pictures from online sources"),
so I haven't bothered with it.
Do you have an idea what typical sizes would be?



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread mamema


mherger wrote: 
> > 
> But you should be able to use MySQL, if you wanted to give it a try. 
> Would be interesting if that would turn out to be faster. One advantage
> 
> certainly would be that it would use another core on today's multi-core
> 
> systems, whereas SQLite is running inside the single LMS thread.

It wasn't clear to me that this is an option. I will try this at least
to report here the outcome. Oh i love the flexibility of this stuff.
Pure fun.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread mamema


Shozzer wrote: 
> I'm not sure that sugar enters into it. More like steroids. When
> TrackStat was enabled for LMS 8.* I immediately installed it, only to
> uninstall it very shortly afterwards - starting and stopping LMS was
> taking too long. Out of curiosity I have installed the Optimizer plugin
> and also TrackStat. Starting and Restarting the server is now extremely
> quick. I did a complete clear and rescan but funnily enough that seemed
> to take longer. I will have to keep an eye on that.
> 

are your files tagged with musicbrainz_id? If so, then you can disable
the musicbrainz id stuff within trackstat settings. Would be interesting
if this is the cause of the slow startb and stop. If so, then you're
number two of those guys which have this. I'm working on a fix for that.
But not ready for primetime.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Shozzer


philchillbill wrote: 
> How much less sugar does it have than the regular non-diet version ?
> [emoji23]
> 
> 
> Sent from my iPhone using Tapatalk

I'm not sure that sugar enters into it. More like steroids. When
TrackStat was enabled for LMS 8.* I immediately installed it, only to
uninstall it very shortly afterwards - starting and stopping LMS was
taking too long. Out of curiosity I have installed the Optimizer plugin
and also TrackStat. Starting and Restarting the server is now extremely
quick. I did a complete clear and rescan but funnily enough that seemed
to take longer. I will have to keep an eye on that.

There is one side effect I am experiencing. The Control Panel refuses to
open. I restarted my Windows 10 PC and tried to open the panel. It
opened straight away. I closed it and tried again a short while after
and now it refuses to open. Any pointers on this please?



Shozzer's Profile: http://forums.slimdevices.com/member.php?userid=58726
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread SpiderJon


Roland0 wrote: 
> DB Optimizer is a plugin for LMS to improve it's database performance
> and memory use.
> More information can be found 'on the homepage'
> (https://www.nexus0.net/pub/sw/lmsdboptim/).

Hi Roland0 

Re: Re-enabling the sqlite_stat4 table in "Other methods to improve
performance" on your 'LMS DB Optimizer info page'
(https://www.nexus0.net/pub/sw/lmsdboptim/).

Where is  Makefile.PL?  And is there anything else you need to do after
uncommenting the line '-DSQLITE_ENABLE_STAT4'  to get things going?

Thanks



Duet x 3, via various hi-fi. LMS on Raspberry Pi OS (Debian 10) on a
Pi4.

SpiderJon's Profile: http://forums.slimdevices.com/member.php?userid=21798
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Michael Herger

Also, because i'm curious and have DB spcialists here, why is the whole
stuff file based and would a dedicated DB server or a Redis server help?


File based: it's simpler and good enough.

We used to ship with MySQL for a while, but maintaining MySQL in the 
background on Windows in a user friendly package wasn't easy.


Redis: isn't a general purpose database, and certainly not SQL.

But you should be able to use MySQL, if you wanted to give it a try. 
Would be interesting if that would turn out to be faster. One advantage 
certainly would be that it would use another core on today's multi-core 
systems, whereas SQLite is running inside the single LMS thread.

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread mamema


Roland0 wrote: 
> Thanks for the feedback.
> 
> I'm still fiddling with some of the parameters (cache sizes, mainly).
> Currently, they should be fine for ~1-2GB free memory.
> However, if enough people with different requirements use this (probably
> unlikely...), it might make sense to add some kind of configuration
> mechanism.
> Another idea would be to add an auto max config, which simply sets the
> caches to the sizes of the databases (although my artwork db currently
> has 1,2GB, so for a small system this may not be such a great idea)

i'm on of those guys willing to throw more RAM on the issue. I'm also
willing to test (with you) how much more RAM is really needed to speed
up the things. I have 32 GB which are more or less usable. But i also
think, there must be a break even point where adding more RAM doesn't
help anymore.

Also, because i'm curious and have DB spcialists here, why is the whole
stuff file based and would a dedicated DB server or a Redis server help?



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread philchillbill


Shozzer wrote: 
> Fair enough, that makes sense. I must be running SQLite!

How much less sugar does it have than the regular non-diet version ?
[emoji23]


Sent from my iPhone using Tapatalk





philchillbill's Profile: http://forums.slimdevices.com/member.php?userid=68920
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Shozzer


mherger wrote: 
> > I am running LMS 8.1.1 on Windows 10. How do I ascertain if I am
> using
> > MySQL or SQLite? There are folders for both within Squeezebox\server.
> 
> If you don't know what you're using, you're using SQLite :-)
> 
> MySQL really only is there for legacy reason. There might be some users
> 
> of it, but it requires manual configuration. Therefore you'd know about
> it.

Fair enough, that makes sense. I must be running SQLite!



Shozzer's Profile: http://forums.slimdevices.com/member.php?userid=58726
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Michael Herger

I am running LMS 8.1.1 on Windows 10. How do I ascertain if I am using
MySQL or SQLite? There are folders for both within Squeezebox\server.


If you don't know what you're using, you're using SQLite :-)

MySQL really only is there for legacy reason. There might be some users 
of it, but it requires manual configuration. Therefore you'd know about it.

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Shozzer


Hi

> Only SQLite is supported, don't install this if your LMS uses MySQL.

I am running LMS 8.1.1 on Windows 10. How do I ascertain if I am using
MySQL or SQLite? There are folders for both within Squeezebox\server.

Thanks.

Steve



Shozzer's Profile: http://forums.slimdevices.com/member.php?userid=58726
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Michael Herger

I'm still fiddling with some of the parameters (cache sizes, mainly).
Currently, they should be fine for ~1-2GB free memory.


I'll have to play with this, in particular with the imageproxy.db, too: 
MAI makes heavy use of it to cache artist artwork etc... Are you using MAI?

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-04 Thread Roland0


philchillbill wrote: 
> I just updated LMS on my Ubuntu NUC to 8.2 and also added your plugin.
> The UI feels much snappier, especially in iPeng with loading thumbnails
> for cover art in the New Music view. Nice work !
> 
> A new and changed rescan of a DB with 70K tracks took 42 seconds. The
> system has an SSD but the music files are on an 8TB HDD.
Thanks for the feedback.

I'm still fiddling with some of the parameters (cache sizes, mainly).
Currently, they should be fine for ~1-2GB free memory.
However, if enough people with different requirements use this (probably
unlikely...), it might make sense to add some kind of configuration
mechanism.
Another idea would be to add an auto max config, which simply sets the
caches to the sizes of the databases (although my artwork db currently
has 1,2GB, so for a small system this may not be such a great idea)



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread philchillbill


I just updated LMS on my Ubuntu NUC to 8.2 and also added your plugin.
The UI feels much snappier, especially in iPeng with loading thumbnails
for cover art in the New Music view. Nice work !

A new and changed rescan of a DB with 70K tracks took 42 seconds. The
system has an SSD but the music files are on an 8TB HDD.





philchillbill's Profile: http://forums.slimdevices.com/member.php?userid=68920
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread mamema


Roland0 wrote: 
> 'gotop' (https://github.com/xxxserxxx/gotop/releases) or 'bottom'
> (https://github.com/ClementTsang/bottom/releases) have binaries for most
> platforms. You could add one of them to your container.

neither of both are working on Synology with the provided binaries. So
i've installed htop in the container. 

I do not see more than one core used. 25% overall and the container is
at 100% if i check within the container itself



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread bobertuk


Roland0 wrote: 
> Could very well be - it's certainly incorrect, even though my local
> installation didn't complain.
> Anyway, I've uploaded a new .zip

That seems to have worked. I guess it was a cache problem.

Thank you



2 x Touch
2 x Radio
2 x Boom
1 x Intel-NUC server/squeezelite running LMS 8.20 (from nightlies) on
Windows 10
1 X Odroid-XU4 server/squeezelite running LMS 7.91 on Ubuntu 16.04
1 x iMac server running macOS Big Sur
WaveIO USB into Lavry DA-10 DAC
Starfish Pre-amp : Based on NAIM NAC 72
Heavily modified NAIM NAP 250 Power-amp
Focal Electra 1027 Be II Speakers

bobertuk's Profile: http://forums.slimdevices.com/member.php?userid=30376
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread manx


last version working for me at least, on debian buster. many thanks for
this :D



manx's Profile: http://forums.slimdevices.com/member.php?userid=70875
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread Roland0


bobertuk wrote: 
> Still the same. I noticed that
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins\Plugins\LMSdbopt/install.xml
> contains 0.0.1
> Could that be the problem? 
Could very well be - it's certainly incorrect, even though my local
installation didn't complain.
Anyway, I've uploaded a new .zip



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread bobertuk


bobertuk wrote: 
> Still the same. I noticed that
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins\Plugins\LMSdbopt/install.xml
> contains 0.0.1
> 
> Could that be the problem? Could the cache at your end contain the wrong
> version?

I just stopped LMS, edited 0.0.1 to
0.0.2, restarted LMS and that seems to have worked,
although I now don't know which version is really running. Is it v0.0.1
or v0.0.2?



2 x Touch
2 x Radio
2 x Boom
1 x Intel-NUC server/squeezelite running LMS 8.20 (from nightlies) on
Windows 10
1 X Odroid-XU4 server/squeezelite running LMS 7.91 on Ubuntu 16.04
1 x iMac server running macOS Big Sur
WaveIO USB into Lavry DA-10 DAC
Starfish Pre-amp : Based on NAIM NAC 72
Heavily modified NAIM NAP 250 Power-amp
Focal Electra 1027 Be II Speakers

bobertuk's Profile: http://forums.slimdevices.com/member.php?userid=30376
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread bobertuk


bobertuk wrote: 
> Still not working. Has downloaded into
> C:\ProgramData\Squeezebox\Cache\InstalledPlugins\Plugins folder but
> hasn't activated. Checkbox to install into plugins still showing so will
> go through cache cleanup and reboot again to see if that helps.

Still the same. I noticed that
C:\ProgramData\Squeezebox\Cache\InstalledPlugins\Plugins\LMSdbopt/install.xml
contains 0.0.1

Could that be the problem? Could the cache at your end contain the wrong
version?



2 x Touch
2 x Radio
2 x Boom
1 x Intel-NUC server/squeezelite running LMS 8.20 (from nightlies) on
Windows 10
1 X Odroid-XU4 server/squeezelite running LMS 7.91 on Ubuntu 16.04
1 x iMac server running macOS Big Sur
WaveIO USB into Lavry DA-10 DAC
Starfish Pre-amp : Based on NAIM NAC 72
Heavily modified NAIM NAP 250 Power-amp
Focal Electra 1027 Be II Speakers

bobertuk's Profile: http://forums.slimdevices.com/member.php?userid=30376
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread bobertuk


Roland0 wrote: 
> My semi-automated deployment re-introduced the typo I fixed yesterday -
> sorry about that. Should work now.

Still not working. Has downloaded into
C:\ProgramData\Squeezebox\Cache\InstalledPlugins\Plugins folder but
hasn't activated. Checkbox to install into plugins still showing so will
go through cache cleanup and reboot again to see if that helps.



2 x Touch
2 x Radio
2 x Boom
1 x Intel-NUC server/squeezelite running LMS 8.20 (from nightlies) on
Windows 10
1 X Odroid-XU4 server/squeezelite running LMS 7.91 on Ubuntu 16.04
1 x iMac server running macOS Big Sur
WaveIO USB into Lavry DA-10 DAC
Starfish Pre-amp : Based on NAIM NAC 72
Heavily modified NAIM NAP 250 Power-amp
Focal Electra 1027 Be II Speakers

bobertuk's Profile: http://forums.slimdevices.com/member.php?userid=30376
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread Roland0


bobertuk wrote: 
> Have just tried updating to v0.0.2 and it fails to update. Have tried
> clearing web browser caches, uninstalling previous version and rebooting
> Windows 10 but no luck and now have no Database Optimizer at all. Any
> help appreciated.
My semi-automated deployment re-introduced the typo I fixed yesterday -
sorry about that. Should work now.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread Roland0


mamema wrote: 
> Sure, but how? I'm a very very poor Synology user (poor means shitty
> interface). As top or htop isn't included in the docker image
'gotop' (https://github.com/xxxserxxx/gotop/releases) or 'bottom'
(https://github.com/ClementTsang/bottom/releases) have binaries for most
platforms. You could add one of them to your container.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread bobertuk


Have just tried updating to v0.0.2 and it fails to update. Have tried
clearing web browser caches, uninstalling previous version and rebooting
Windows 10 but no luck and now have no Database Optimizer at all. Any
help appreciated.



2 x Touch
2 x Radio
2 x Boom
1 x Intel-NUC server/squeezelite running LMS 7.92 (from nightlies) on
Windows 10
1 X Odroid-XU4 server/squeezelite running LMS 7.91 on Ubuntu 16.04
1 x iMac server running macos Hich Sierra
WaveIO USB into Lavry DA-10 DAC
Starfish Pre-amp : Based on NAIM NAC 72
Heavily modified NAIM NAP 250 Power-amp
Behringer DEQ2496
Linn Isobarik DMS

bobertuk's Profile: http://forums.slimdevices.com/member.php?userid=30376
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread mamema


Roland0 wrote: 
> 
> Pretty confident that'll overwhelmingly be the new indices.
> However, this may actually be a scenario where multi-threading could be
> used, so mamema could check if more than one core is utilized the next
> time a trackstat refresh runs.
> 

Sure, but how? I'm a very very poor Synology user (poor means shitty
interface). As top or htop isn't included in the docker image, i can
only see that 25% CPU is used, which means with my 4-core NAS it use 1
core with 100%



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread Roland0


mherger wrote: 
> 
> From my poor memory I remember SQLite caches the index, not the data
> itself.
> 

actually, it does cache the data:
SQLite docs wrote: 
> 
> maximum number of database disk pages that SQLite will hold in memory 
> 

mherger wrote: 
> 
> Therefore assigning a lot of data to the artwork cache might do no harm
> in most cases, but promise more than it delivers? 
> 
Verifying that caching artworkdb works is fairly simple:
- LMS process size grows larger the more covers have been displayed
- my artworkdb is on a HDD, so I can literally hear that it's accessed
the first time a specific cover is displayed, and not on subsequent
renderings

> 
> trackstat
> Would be interesting to know which tweak makes the difference, or
> whether it's really the combination of them all.
> 
Pretty confident that'll overwhelmingly be the new indices.
However, this may actually be a scenario where multi-threading could be
used, so mamema could check if more than one core is utilized the next
time a trackstat refresh runs.

I've also released a new version which adds a dropExtraTrackStatIndices
API command, which could be helpful if anyone really wants to perform a
detailed analysis.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread mamema


mherger wrote: 
> > My, i admit rather short test reveals, that the trackscan scan task
> went
> 
> I guess that should read "trackstat"?
> 

Yes!



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-03 Thread Michael Herger

My, i admit rather short test reveals, that the trackscan scan task went


I guess that should read "trackstat"?


don't from 3 hours to 4 min on LMS restart and from 3 hours to 15 min on
forced rescan. Even though modifying the SELECT statement to prevent
musicbrainz_id duplicates will bring the scanning down to 4 secs each,
so..


Well, that's a good result :-). Would be interesting to know which tweak 
makes the difference, or whether it's really the combination of them all.



Another improvement is in the webinterface everywhere where cover images
needs to be displayed it "seems" faster.


Ok, I'll see what I can learn from that plugin. Thanks for the data point!
___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread mamema


mherger wrote: 
> > Anything accessing the DB should benefit.
> 
> That's rather general... where would I as a user see improvements? Not 
> in a benchmark, but in LMS use.
> .

My, i admit rather short test reveals, that the trackscan scan task went
don't from 3 hours to 4 min on LMS restart and from 3 hours to 15 min on
forced rescan. Even though modifying the SELECT statement to prevent
musicbrainz_id duplicates will bring the scanning down to 4 secs each,
so..

Another improvement is in the webinterface everywhere where cover images
needs to be displayed it "seems" faster.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Michael Herger

Anything accessing the DB should benefit.


That's rather general... where would I as a user see improvements? Not 
in a benchmark, but in LMS use.


Eg. the scanner has totally different requirements than the server: 
scanner is mostly writes, server is almost read-only. From my poor 
memory I remember SQLite caches the index, not the data itself. 
Therefore assigning a lot of data to the artwork cache might do no harm 
in most cases, but promise more than it delivers? A lot of the heavy 
queries are cached by the application. Optimizing the cache's DB might 
be more important than the library's etc.


___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread SpiderJon

Roland0 wrote: 
> I think I've found the issue. Try removing the repository, restart, add
> it again, active the plugin, and restart (and report if there are any
> issues remaining). Thanks.

Yep, that's fixed it.

I've also checked that mmap is enabled —  it is:


$ sudo lsof |grep mem-r|grep 'persist.db$\|library.db$'
squeezebo 10581squeezeboxserver  mem-r REG 
179,2 285020160 265905 /var/lib/squeezeboxserver/cache/persist.db
squeezebo 10581squeezeboxserver  mem-r REG 
179,2 217821184 265904
/var/lib/squeezeboxserver/cache/library.db

Thanks.



Duet x 3, via various hi-fi. LMS on Raspberry Pi OS (Debian 10) on a
Pi4.

SpiderJon's Profile: http://forums.slimdevices.com/member.php?userid=21798
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0


mherger wrote: 
> > DB Optimizer is a plugin for LMS to improve it's database performance
> > and memory use.
> > More information can be found 'on the homepage'
> > (https://www.nexus0.net/pub/sw/lmsdboptim/).
> 
> Would you have a rough idea what parts of LMS' functionality would take
> 
> advantage of which tweak? As Erland said it might be interesting to 
> "backport" some of these to the main line product.

Anything accessing the DB should benefit.
I've listed the various tweaks on the homepage (with links to the
relevant SQLite docs):
Some correct oversights (e.g. library uses temp_store = MEMORY, persist
does not; artwork always get 80MB cache even at "Maximum") or bit rot
(e.g. I suspect the 4GB DB cache is a result of SQLite switching from
1KB pages to 4KB some years ago).
Others are IMHO no-brainers (PRAGMA optimize before shutdown,
multi-threading)
Database cache sizes are really unbalanced (2*8MB+1MB -> 3*80MB ->
4GB+80MB)
mmap might be more complex to assess, but for anything not Windows is
seems great (25% less memory on my setup, plus (probably) better
performance)



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0

mamema wrote: 
> 
> But as I said, you are doing more than index creating. I would be happy
> to test, unfortunately the plugin doesm’t Show up after reboot.
It seems I initially had a typo in the repo.xml. However, at least on my
system, it now works.
If removing the repository, restarting, adding it again, activating the
plugin, and restarting doesn't work, it would be helpful if you could
check the server logs.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread mamema

Roland0 wrote: 
> see previous comment for the rationale. 
> _id=?)



You are doing more with your plugin than creating indexes, which might be
good, not tested yet.
But, dunno if it is an issue with pre 8.2 versions, because the indexes
were created fine within trackstat without your plugin.
Additionally i am experimenting with additions, so duplicate
musicbrainz_id didn’t slow down trackstat.
With this addition, trackstat url compare from temp table isn’t an issue
anymore. Was running three hours.

But as I said, you are doing more than index creating. I would be happy to
test, unfortunately the plugin doesm’t Show up after reboot.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Michael Herger

DB Optimizer is a plugin for LMS to improve it's database performance
and memory use.
More information can be found 'on the homepage'
(https://www.nexus0.net/pub/sw/lmsdboptim/).


Would you have a rough idea what parts of LMS' functionality would take 
advantage of which tweak? As Erland said it might be interesting to 
"backport" some of these to the main line product.

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0


Well, talk about a derail ;-)

So, to recap:
- the purpose of this plugin is to apply some tuning to the database
which should be generally beneficial (and is unrelated to TrackStat,
although TrackStat, using the DB heavily, may benefit as well)
- I added some indices to my database for TrackStat a while ago. Since I
had them already available, I thought I might as well include them in
the plugin (they will be only created if TrackStat is installed).
- TrackStat's original indices are included as well, since some were
missing from my DB for unknown reasons, so it's possible they are
missing from other TrackStat users' DBs as well  (they will be only
created if they are missing).



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0


mamema wrote: 
> which indexes do you mean? Except those from the temp table, we've
> discussed in the Dev Area, all needed indexes are there.
> Trackstat creates and deletes indexes and both is working.
see previous comment for the rationale. also:


Code:

explain query plan UPDATE track_history SET url=(select tracks.url from 
library.tracks where tracks.musicbrainz_id is not null and 
tracks.musicbrainz_id=track_history.musicbrainz_id and 
track_history.urlmd5!=tracks.urlmd5 and length(tracks.url)<512),urlmd5=(select 
tracks.urlmd5 from tracks where tracks.musicbrainz_id is not null and 
tracks.musicbrainz_id=track_history.musicbrainz_id and 
track_history.urlmd5!=tracks.urlmd5 and length(tracks.url)<512) where exists 
(select tracks.url from tracks where tracks.musicbrainz_id is not null and 
tracks.musicbrainz_id=track_history.musicbrainz_id and track_history.urlmd5 != 
tracks.urlmd5 and length(tracks.url)<512);




Code:

SCAN TABLE track_history
  CORRELATED SCALAR SUBQUERY 3
  SEARCH TABLE tracks USING COVERING INDEX trackStatMBURLIndex 
(musicbrainz_id=?)
  CORRELATED SCALAR SUBQUERY 1
  SEARCH TABLE tracks USING COVERING INDEX trackStatMBURLIndex 
(musicbrainz_id=?)
  CORRELATED SCALAR SUBQUERY 2
  SEARCH TABLE tracks USING COVERING INDEX trackStatMBURLIndex 
(musicbrainz_id=?)




Code:

explain query plan select library.tracks.url from 
library.tracks,persist.track_history where library.tracks.musicbrainz_id is not 
null and library.tracks.musicbrainz_id=track_history.musicbrainz_id and 
persist.track_history.urlmd5 != library.tracks.urlmd5 and 
length(library.tracks.url)<512;




Code:


  SCAN TABLE track_history USING COVERING INDEX tsurlmdmbIndex
  SEARCH TABLE tracks USING COVERING INDEX trackStatMBURLIndex 
(musicbrainz_id=?)




'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread erland


Roland0 wrote: 
> 
> Which indices do you mean? The ones taken from TrackStat should use the
> same names afaik.
> 
My mistake, I accidentally looked at the MySQL indexes TrackStat
creates. The SQLite indexes are named the same so no duplicates, sorry
for the confusion regarding this.



Erland Isaksson ('My homepage' (http://erland.isaksson.info))
Developer of 'many plugins/applets'
(https://wiki.slimdevices.com/index.php/User_Erland.html)
*Starting with LMS 8.0 I no longer support my plugins/applets* ('see
here for more information'
(https://forums.slimdevices.com/showthread.php?49483-Announce-New-versions-of-erlands-plugins&p=998836&viewfull=1#post998836)
)

erland's Profile: http://forums.slimdevices.com/member.php?userid=3124
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread erland

mamema wrote: 
> which indexes do you mean? Except those from the temp table, we've
> discussed in the Dev Area, all needed indexes are there.
> Trackstat creates and deletes indexes and both is working.

I read the code a bit fast, the naming is the same so there aren’t any
duplicated indexes, I just wonder why this plugin creates indexes that
TrackStat already have created, gives me a feeling something doesnt work
properly in TrackStat.

The following should be created by TrackStat:

Code:


  CREATE INDEX IF NOT EXISTS persistentdb.tsurlIndex on track_statistics (url);
  CREATE INDEX IF NOT EXISTS persistentdb.tsurlmd5Index on track_statistics 
(urlmd5);
  CREATE INDEX IF NOT EXISTS persistentdb.tsmusicbrainzIndex on 
track_statistics (musicbrainz_id);
  CREATE INDEX IF NOT EXISTS persistentdb.tshurlIndex on track_history (url);
  CREATE INDEX IF NOT EXISTS persistentdb.tshmusicbrainzIndex on track_history 
(musicbrainz_id);
  CREATE INDEX IF NOT EXISTS trackStatMBIndex on tracks (musicbrainz_id)
  



The following is created by this plugin (if I read the code properly)

Code:


  CREATE INDEX IF NOT EXISTS persistentdb.tsurlIndex on track_statistics (url)
  CREATE INDEX IF NOT EXISTS persistentdb.tsurlmd5Index on track_statistics 
(urlmd5)
  CREATE INDEX IF NOT EXISTS persistentdb.tsmusicbrainzIndex on 
track_statistics (musicbrainz_id)
  CREATE INDEX IF NOT EXISTS persistentdb.tshurlIndex on track_history (url)
  CREATE INDEX IF NOT EXISTS persistentdb.tshmusicbrainzIndex on track_history 
(musicbrainz_id)
  CREATE INDEX IF NOT EXISTS trackStatMBIndex on tracks (musicbrainz_id)
  CREATE INDEX IF NOT EXISTS persistentdb.tsurlmdmbIndex on 
track_history(urlmd5,musicbrainz_id)
  CREATE INDEX IF NOT EXISTS persistentdb.tsurlmdmbStatIndex on 
track_statistics(urlmd5,musicbrainz_id)
  CREATE INDEX IF NOT EXISTS trackStatMBURLIndex on tracks 
(musicbrainz_id,urlmd5,url)
  



If the extra 3 indexes this plugin creates improve TrackStat performance
it would be great if someone have the time to release a new version of
TrackStat that includes them.

And the tracks table trackStatMBIndex index should really be created by
core LMS if it helps performance since core LMS owns the tracks table.



Erland Isaksson ('My homepage' (http://erland.isaksson.info))
Developer of 'many plugins/applets'
(https://wiki.slimdevices.com/index.php/User_Erland.html)
*Starting with LMS 8.0 I no longer support my plugins/applets* ('see
here for more information'
(https://forums.slimdevices.com/showthread.php?49483-Announce-New-versions-of-erlands-plugins&p=998836&viewfull=1#post998836)
)

erland's Profile: http://forums.slimdevices.com/member.php?userid=3124
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0

erland wrote: 
> Hopefully someone will take the time to update TrackStat so it creates
> the indexes by itself. 
> 
Would certainly be a better solution.

> The intention is that TrackStat should already today create some of the
> single column indexes but maybe that doesn’t work since you have them
> in your plugin ?
I've been using TrackStat for a long time, and quite a while ago (years)
I noticed that some of the default indices were missing - no idea why.
Back then, I also added some other indices. Since I was writing this
plugin anyway, I decided to include them - since I use 'IF NOT EXISTS',
existing ones won't be affected.

> 
> As it’s done now I think the indexes will be duplicated since you use
> different index names, duplicate indexes might decrease write
> performance.
Which indices do you mean? The ones taken from TrackStat should use the
same names afaik.

> The index in tracks table should probably be added to LMS code instead
> of having it as a plugin. The reason TrackStat creates an index in
> tracks table is just because it was harder to convince/get core
> developer to do additions to the core 15 years ago, should be easier
> today when LMS development is driven by community.
I guess Michael would be the person to assess which index is of general
benefit.

> Have you performed any tests to see which TrackStat operations that are
> improved ?
> 
Back then, I probably did, but that was a couple of years ago, so I'm
rather hazy on the details.

> Having them in your plugin as a first step could be a good idea to
> verify how much improvement it results
Generally, maybe, but since the plugin's main purpose is to change
various DB parameters, so it'll be difficult to isolate specifics.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread mamema

erland wrote: 
> Hopefully someone will take the time to update TrackStat so it creates
> the indexes by itself. The intention is that TrackStat should already
> today create some of the single column indexes but maybe that doesn’t
> work .

which indexes do you mean? Except those from the temp table, we've
discussed in the Dev Area, all needed indexes are there.
Trackstat creates and deletes indexes and both is working.



mamema's Profile: http://forums.slimdevices.com/member.php?userid=44662
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0


mherger wrote: 
> >
> /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/LMSdboptim/LMSdbopt/lib
> 
> That's one folder too many. The plugin archive should extract without a
> 
> sub-folder for Plugins.pm.
It extracts to LMSdbopt/Plugin.pm, which I think is correct. I think the
issue was in repo.xml (plugin name="LMSdboptim" instead of plugin
name="LMSdbopt")



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread erland

Hopefully someone will take the time to update TrackStat so it creates
the indexes by itself. The intention is that TrackStat should already
today create some of the single column indexes but maybe that doesn’t
work since you have them in your plugin ? As it’s done now I think the
indexes will be duplicated since you use different index names,
duplicate indexes might decrease write performance.

The index in tracks table should probably be added to LMS code instead
of having it as a plugin. The reason TrackStat creates an index in
tracks table is just because it was harder to convince/get core
developer to do additions to the core 15 years ago, should be easier
today when LMS development is driven by community.

Have you performed any tests to see which TrackStat operations that are
improved ?

Having them in your plugin as a first step could be a good idea to
verify how much improvement it results in, so I’m not criticizing your
work just mentioning the above since I think it’s the next step to take.



Erland Isaksson ('My homepage' (http://erland.isaksson.info))
Developer of 'many plugins/applets'
(https://wiki.slimdevices.com/index.php/User_Erland.html)
*Starting with LMS 8.0 I no longer support my plugins/applets* ('see
here for more information'
(https://forums.slimdevices.com/showthread.php?49483-Announce-New-versions-of-erlands-plugins&p=998836&viewfull=1#post998836)
)

erland's Profile: http://forums.slimdevices.com/member.php?userid=3124
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0


SpiderJon wrote: 
> [[21-03-02 18:27:09.6876] main::init (388) Starting Logitech Media
> Server (v8.1.1, 1610364019, Thu Jan 14 06:24:07 CET 2021) perl 5.028001
> - arm-linux-gnueabihf-thread-multi-64int
> [21-03-02 18:27:11.4689] Slim::bootstrap::tryModuleLoad (286) Warning:
> Module [Plugins::LMSdbopt::Plugin] failed to load:
> Can't locate Plugins/LMSdbopt/Plugin.pm in @INC (you may need to install
> the Plugins::LMSdbopt::Plugin module) (@INC contains:
> /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/LMSdboptim/LMSdbopt/lib
> /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/lib
> /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomBrowse/lib
> /var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/BBCSounds/lib
> /var/lib/squeezeboxserver/cache/InstalledPlugins
> /usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int
> /usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/auto
> /usr/share/squeezeboxserver/CPAN/arch/5.28.1/arm-linux-gnueabihf-thread-multi-64int
> /usr/share/squeezeboxserver/CPAN/arch/5.28.1/arm-linux-gnueabihf-thread-multi-64int/auto
> /usr/share/squeezeboxserver/CPAN/arch/arm-linux-gnueabihf-thread-multi-64int
> /usr/share/squeezeboxserver/CPAN/arch/5.28
> /usr/share/squeezeboxserver/lib /usr/share/squeezeboxserver/CPAN
> /usr/share/squeezeboxserver /usr/sbin /etc/perl
> /usr/local/lib/arm-linux-gnueabihf/perl/5.28.1
> /usr/local/share/perl/5.28.1 /usr/lib/arm-linux-gnueabihf/perl5/5.28
> /usr/share/perl5 /usr/lib/arm-linux-gnueabihf/perl/5.28
> /usr/share/perl/5.28 /usr/local/lib/site_perl
> /usr/lib/arm-linux-gnueabihf/perl-base) at (eval 1056) line 1.
> BEGIN failed--compilation aborted at (eval 1056) line 1.
> 

I think I've found the issue. Try removing the repository, restart, add
it again, active the plugin, and restart (and report if there are any
issues remaining). Thanks.



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Michael Herger

/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/LMSdboptim/LMSdbopt/lib


That's one folder too many. The plugin archive should extract without a 
sub-folder for Plugins.pm.


___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread SpiderJon


Roland0 wrote: 
> Anything in the server logs?

*[21-03-02 18:27:09.6876] main::init (388) Starting Logitech Media
Server (v8.1.1, 1610364019, Thu Jan 14 06:24:07 CET 2021) perl 5.028001
- arm-linux-gnueabihf-thread-multi-64int
[21-03-02 18:27:11.4689] Slim::bootstrap::tryModuleLoad (286) Warning:
Module [Plugins::LMSdbopt::Plugin] failed to load:
Can't locate Plugins/LMSdbopt/Plugin.pm in @INC (you may need to install
the Plugins::LMSdbopt::Plugin module) (@INC contains:
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/LMSdboptim/LMSdbopt/lib
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/lib
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomBrowse/lib
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/BBCSounds/lib
/var/lib/squeezeboxserver/cache/InstalledPlugins
/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int
/usr/share/squeezeboxserver/CPAN/arch/5.28/arm-linux-gnueabihf-thread-multi-64int/auto
/usr/share/squeezeboxserver/CPAN/arch/5.28.1/arm-linux-gnueabihf-thread-multi-64int
/usr/share/squeezeboxserver/CPAN/arch/5.28.1/arm-linux-gnueabihf-thread-multi-64int/auto
/usr/share/squeezeboxserver/CPAN/arch/arm-linux-gnueabihf-thread-multi-64int
/usr/share/squeezeboxserver/CPAN/arch/5.28
/usr/share/squeezeboxserver/lib /usr/share/squeezeboxserver/CPAN
/usr/share/squeezeboxserver /usr/sbin /etc/perl
/usr/local/lib/arm-linux-gnueabihf/perl/5.28.1
/usr/local/share/perl/5.28.1 /usr/lib/arm-linux-gnueabihf/perl5/5.28
/usr/share/perl5 /usr/lib/arm-linux-gnueabihf/perl/5.28
/usr/share/perl/5.28 /usr/local/lib/site_perl
/usr/lib/arm-linux-gnueabihf/perl-base) at (eval 1056) line 1.
BEGIN failed--compilation aborted at (eval 1056) line 1.

[21-03-02 18:27:11.4699] Slim::Utils::PluginManager::load (323) Error:
Couldn't load Plugins::LMSdbopt::Plugin
[21-03-02 18:27:11.7114] Plugins::CustomScan::Scanner::initDatabase
(114) CustomScan: Creating database tables
[21-03-02 18:27:11.7133] Plugins::CustomScan::Scanner::initDatabase
(119) CustomScan: Upgrading database adding table column valuesort,
please wait...
[21-03-02 18:27:11.7139] Plugins::CustomScan::Scanner::executeSQLFile
(522) Couldn't open:
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/SQL/SQLite/dbupgrade_valuesort.sql
: No such file or directory
[21-03-02 18:27:11.7143] Plugins::CustomScan::Scanner::initDatabase
(125) CustomScan: Upgrading database adding table column extravalue,
please wait...
[21-03-02 18:27:11.7149] Plugins::CustomScan::Scanner::executeSQLFile
(522) Couldn't open:
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/SQL/SQLite/dbupgrade_extravalue.sql
: No such file or directory
[21-03-02 18:27:11.7154] Plugins::CustomScan::Scanner::initDatabase
(131) CustomScan: Upgrading database adding table column valuetype,
please wait...
[21-03-02 18:27:11.7159] Plugins::CustomScan::Scanner::executeSQLFile
(522) Couldn't open:
/var/lib/squeezeboxserver/cache/InstalledPlugins/Plugins/CustomScan/SQL/SQLite/dbupgrade_valuetype.sql
: No such file or directory

*



Duet x 3, via various hi-fi. LMS on Raspberry Pi OS (Debian 10) on a
Pi4.

SpiderJon's Profile: http://forums.slimdevices.com/member.php?userid=21798
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0

SpiderJon wrote: 
> 
> However, I followed the installation routine —
> — but "DB Optimizer" didn't get installed.
> 
> So I tried activating the plugin (ie, ticking the box), saving settings
> and restarting LMS three more times, but still no luck installing it —
> it's just showing as an unticked box in its own "LMS DB Optimizer
> plugin" section of the Plugins page.
> 

Anything in the server logs?



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


Re: [SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread SpiderJon

Roland0 wrote: 
> DB Optimizer is a plugin for LMS to improve its database performance and
> memory use.
> More information can be found 'on the homepage'
> (https://www.nexus0.net/pub/sw/lmsdboptim/).

Very interested in trying this.

However, I followed the installation routine —


In the LMS web interface in Settings > Plugins, add
https://www.nexus0.net/pub/sw/lmsdboptim/repo.xml to the list of
Additional Repositories, activate the plugin and re-start LMS.

— but "DB Optimizer" didn't get installed.

So I tried activating the plugin (ie, ticking the box), saving settings
and restarting LMS three more times, but still no luck installing it —
it's just showing as an unticked box in its own "LMS DB Optimizer
plugin" section of the Plugins page.

I'm using 

LMS Version: 8.1.1 - 1610364019 
Operating system: Debian - EN - utf8
Platform Architecture: armv7l-linux
Perl Version: 5.28.1 - arm-linux-gnueabihf-thread-multi-64int
Audio::Scan: 1.02
IO::Socket::SSL: 2.060
Database Version: DBD::SQLite 1.58 (sqlite 3.22.0)

Thanks for any help.



Duet x 3, via various hi-fi. LMS on Raspberry Pi OS (Debian 10) on a
Pi4.

SpiderJon's Profile: http://forums.slimdevices.com/member.php?userid=21798
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins


[SlimDevices: Plugins] [Announce] Database Optimizer plugin

2021-03-02 Thread Roland0


DB Optimizer is a plugin for LMS to improve it's database performance
and memory use.
More information can be found 'on the homepage'
(https://www.nexus0.net/pub/sw/lmsdboptim/).



'Various SW' (https://www.nexus0.net/pub/sw/): Web Interface | TUI |
Playlist Editor / Generator | Music Classification | Similar Music |
Announce | EventTrigger | Chiptunes | LMSlib2go | ...
'Various HowTos' (https://www.nexus0.net/pub/documents/LMS/): build a
self-contained LMS | Bluetooth/ALSA | Control LMS with any device | ...

Roland0's Profile: http://forums.slimdevices.com/member.php?userid=56808
View this thread: http://forums.slimdevices.com/showthread.php?t=114067

___
plugins mailing list
plugins@lists.slimdevices.com
http://lists.slimdevices.com/mailman/listinfo/plugins