Re: [Dorset] Wireless dropping out
I haven't been following this, but have you checked for a firmware update for the router that might fix the issue? Hamish On 20/02/2021 12:56, CA Wills wrote: > Started a new email on this subject due to delays in replying. We have > been relaying our patio during the week between rain showers, so have > not had a clear time slot to 'play'. > > Update today. 1200hrs. > > Removed all connections from the TT Super(?) Router and replaced using > the very old previous Huawei HG633 router. > > After reconnecting to old HG633 router and checking I had wireless set > up, it showed that Lily's Tablet had connected automatically via > wireless. > > Then set up my laptop (wireless) and it connected OK; moved into the > Living room and still connected; proved that the _laptop_ wireless is > OK. Checked speed and appears no loss in speed. > > Conclusion is that something on the 'new' router has upset 'my' link. > As we need the wireless connection for tomorrow morning, I'm staying > with the 'old' router till after lunch. > > Next stage is to reconnect the 'new' router after setting it back to > default, will then check if wireless is OK again. Hoping that may > clear whatever upset my wireless. > > If all goes well then end of problem. If I still can't connect the > laptop but all else is OK will get onto TT, report the problem and ask > for a replacement router (can but hope!). As Peter reported a while > back he had a similar problem with one item being rejected connection, > I believe that was also with a TT router. > > Thanks for all the suggestions and help, will email when done > reconnecting tomorrow. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] stdout and stderr redirection weirdness
Thanks, this all sounds like it'll be a very good idea. syslog is attractive, but I'm not sure this even has syslog. Or maybe I need to turn it on, I remember hearing something about it anyway. I'll figure it out and get back to you. At the moment I'm adding indexes to the database so it'll have to be a bit later on - this looks like it will take a few hours. Hamish On 18/02/2021 16:20, Keith Edmunds wrote: > OK, this is what I would do. Check whether logger (typically > /usr/bin/loggger) is installed on the system. If it is, create a script > that runs at boot as follows: > > #!/bin/ash [if /bin/bash is available, I'd use that] > > /usr/bin/logger -t xyzzy "SHELL=$SHELL" > /usr/bin/logger -t xyzzy "me=$(whoami)" > echo "test" > /tmp/me.log > /usr/bin/logger -t xyzzy "Status after file write: $?" > > What that does: > > - /usr/bin/logger makes entries in syslog, so no need to worry about >writing files > > - "-t xyzzy" will tag each syslog entry with "xyzzy". Of course you can >use any string, but that allows you to 'grep xyzzy /var/log/messages' >(or wherever syslog writes) > > - we check the shell (is it really ash?) > > - we check who we are (are we really root?) > > - we try writing to a file and report the status of doing so > > Just looking at your original post, the other thing I'd change is the > relative file reference. Rather than writing to ../stdout.log, just write > to /tmp/stdout.log. Better still: > > python3 ./main.py --id "NAS" 2>&1 | tee /usr/bin/logger -t xyzzy > > ...and have it sent to syslog. > > hth signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] stdout and stderr redirection weirdness
On 18/02/2021 15:34, Keith Edmunds wrote: >> I'm finding that during startup piping and redirecting output doesn't >> work. This is using the ash shell from busybox. > By "during startup" do you mean system startup or your process startup? During system startup (that's the only way to get it running automatically). It piggybacks of the startup scripts for the webserver. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] stdout and stderr redirection weirdness
Hi all, I have yet another query relating to the NAS box. I suspect this issue is once again due to the weird setup and ancient software. I'm finding that during startup piping and redirecting output doesn't work. This is using the ash shell from busybox. I have the following in my script: python3 ./main.py --id "NAS" 2>&1 | tee ../stdout.log I have also tried simple redirects with > ../stdout.log, and | cat > ../stdout.log The file is created but always remains empty regardless of what I do. It does work when I SSH in and run commands though. Any ideas what might be causing this? I've never encountered this behaviour before. Thanks, Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] On Second Entry to MP3 Player Function Program Stops Responding to External Commands
Yep, I saw it. I'm just finishing off my assignment, but should be able to look at several WMT things this afternoon with luck. Hamish On 18/02/2021 10:08, Terry Coles wrote: > On Thursday, 18 February 2021 09:48:20 GMT Hamish McIntyre-Bhatty wrote: >> Can't recall if you've tried this, but what if you replace the call to >> mpg123 with a call to echo or similar, just to narrow down whether it >> might be mpg123 that's causing the issue. >> >> If it only crashes with mpg123, perhaps there are some verbose or debug >> flags you can use for mpg123 to reveal more information? > I only just got this, despite the datestamp being 7 minutes earlier than my > post. > > Have you seen my latest? > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] On Second Entry to MP3 Player Function Program Stops Responding to External Commands
Hi Terry, Can't recall if you've tried this, but what if you replace the call to mpg123 with a call to echo or similar, just to narrow down whether it might be mpg123 that's causing the issue. If it only crashes with mpg123, perhaps there are some verbose or debug flags you can use for mpg123 to reveal more information? Hamish On 17/02/2021 13:32, Terry Coles wrote: > On Tuesday, 16 February 2021 15:25:28 GMT Terry Coles wrote: >> So I'm kind of back where I started. > This morning I tried a variety of things to solve this; still to no avail. > > I've attached two more code fragments to this message; these are copied from > the latest version of the software on the Pis. The file > Minster_Bells_Software_Fragment.txt shows what I have on the Bells Pi to play > Change Rings on demand. Minster_Music_Software_Fragment2.txt shows the music > player version of this; which exhibits this problem. > > In order to get to the bottom of this I have: > > 1. Reduced the number of MP3 files in the Playlist directory to 1. > > 2. Re-written the code so that the line that uses glob to parse the files in > the directory is replaced with the actual path and filename to the first file. > > 3. Re-written the code to remove the while loop, to simplify things. > > 4. Commented out the lines that invoked the mpg123 player and terminated it > in the mp3_player_stop() function. > > Items 1 to 3 made no difference whatsoever to the behaviour; I still got the > freezing when I tried to restart the player/ > > With Item 4 the messages from the Webserver were responded to reliably but of > course no music was played. > > It would appear therefore that once the mpg123 player has been invoked and > then terminated, invoking it a second time makes the overall program > unresponsive. This I cannot understand because the "Music Playing" message > is > being printed, so presumably the lock up occurs at the line containing: > > mp3_wait = mp3_player.wait() > > I realise that the program will halt at that point until the mpg123 player > terminates, but the commands coming in are in new threads so they should > still > be responded to. (They have once.) > > In any case, the Bells Pi code uses the same approach, so how come that works > and the Music code doesn't? > > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] QEMU emulation features
Hi there, I'm having some trouble finding any information on this, so I was wondering if any of you happen to know if QEMU can emulate SMART (specifically the ATA/NVME secure erase command), and/or OPAL drive encryption (a common feature with SSDs, unlocked using a PSID - Physical Security ID - to disable encryption). I'm assuming the answer is no to both of these things, but worth asking. I can of course try them, but I don't have the very latest version of qemu installed so I figure it's worth asking here in case anyone already knows. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] On Second Entry to MP3 Player Function Program Stops Responding to External Commands
On 16/02/2021 12:15, Terry Coles wrote: > On Tuesday, 16 February 2021 12:03:28 GMT Hamish McIntyre-Bhatty wrote: >> Well I must say, I'm not that familiar with apscheduler, but it looks to >> me as if it's causing more issues and complexity than threading would, >> at least in some cases. >> >> I also don't think it exactly avoid threading, because apscheduler is >> probably using its own internal threading to achieve the results. > I didn't want to avoid threading as such; I was trying to ensure that all > instances of mp3_player_start() were running in a thread as per the > suggestion > by Patrick. As it happens that hasn't solved the original problem. > >> If I wrote this, I'd be more likely to use multiple threads and some >> global state to control them - much like how I've done in the river >> system with variables like config.EXITING to signal program shutdown. > I suspect you are right, but apscheduler has worked well for me for a long > time and is used extensively in this software to schedule events at certain > times of the day and at intervals down to 1 second (to trigger the check for > messages in the sockets code). > > It may be that my problem here is nothing to do with threading and something > else is blocking the minstermusic execution from continuing when I try to > start or restart the music. If that is the case I'll revert the code to the > way it was because that was working fine apart from this current problem. That is very possible. I should add that just because I'd think to do it that way doesn't mean that that's necessarily the right way to do it - you could also consider it reinventing the wheel. If I find some time, I'll look into this for you, but I figure the issue you posted on the forum is higher priority? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] On Second Entry to MP3 Player Function Program Stops Responding to External Commands
On 16/02/2021 11:59, Terry Coles wrote: > On Tuesday, 16 February 2021 10:50:59 GMT Terry Coles wrote: >> If anyone wants to look at the full package the code is on our GitLab >> Repository at: >> >> https://gitlab.com/wmtprojectsteam/minster_bells_re-engineering > BTW. I've now realised that I introduced (yet another) bug when I did the > last update. At the end of the file I wrote: > > if opening_hours: # Program has begun after 10 am, > sched.add_job(mp3_player_start, 'interval', seconds=10, > max_instances=1, id='First Start on Power Up') > > what I was trying to do was to get apscheduler to fire off the function > mp3_player_start ten seconds after the initial startup code had completed, > but > what I've actually done is triggered it every 10 secs with an instance max of > 1, which barfs. ;-( > > Is there a way to do this or should I simply forget about trying to do it > with > apscheduler? Well I must say, I'm not that familiar with apscheduler, but it looks to me as if it's causing more issues and complexity than threading would, at least in some cases. I also don't think it exactly avoid threading, because apscheduler is probably using its own internal threading to achieve the results. If I wrote this, I'd be more likely to use multiple threads and some global state to control them - much like how I've done in the river system with variables like config.EXITING to signal program shutdown. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Microsoft on Raspberry Pi
Is Raspbian stretch still supported? Last I heard they don't really support anything except the latest version of Raspbian. Hamish On 06/02/2021 15:17, Andrew wrote: > I can confirm that this problem does exist with Raspbian Buster, even > on the lite version: > > Preparing to unpack .../14-raspberrypi-sys-mods_20210125_armhf.deb ... > Unpacking raspberrypi-sys-mods (20210125) over (20201026) ... > Setting up raspberrypi-sys-mods (20210125) ... > Adding vscode repo... > > > $ cat /etc/apt/sources.list.d/vscode.list > ### THIS FILE IS AUTOMATICALLY CONFIGURED ### > # You may comment out this entry, but any other modifications may be > lost. > deb [arch=amd64,arm64,armhf] http://packages.microsoft.com/repos/code > stable main > > /etc/apt/trusted.gpg.d/microsoft.gpg got created. > > The problem doesn't seem to exist on Raspbian Stretch, at least not yet. > > Microsoft bought GitHub. Or "Microsoft GitHub" as it should be called > now. I don't think they can buy Git its self as it is open-source > software. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] HP Printers and Flash
I actually didn't know there was a web interface. Do I need to do anything to secure that? Hamish On 06/02/2021 11:09, Tim wrote: > I am using a Laser jet Pro M277 (Scanner copier printer) and it loads > the web interface for me OK. > > > Tim H > > On 06/02/2021 10:59, Hamish McIntyre-Bhatty wrote: >> Not for me at least - HPLIP-GUI is Qt based and works with CUPS, not had >> any issues. >> >> Hamish >> >> On 06/02/2021 10:57, PeterMerchant wrote: >>> In a discussion with friends yesterday, they discussed the problems of >>> communicating with HP printers from W10 now that Flash has been >>> killed, because apparently HP drivers use it. Is this a problem from >>> Linux? I expect not, but if I come across a printer it would be good >>> to know. >>> >>> Peter >>> >>> >>> signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] HP Printers and Flash
Not for me at least - HPLIP-GUI is Qt based and works with CUPS, not had any issues. Hamish On 06/02/2021 10:57, PeterMerchant wrote: > In a discussion with friends yesterday, they discussed the problems of > communicating with HP printers from W10 now that Flash has been > killed, because apparently HP drivers use it. Is this a problem from > Linux? I expect not, but if I come across a printer it would be good > to know. > > Peter > > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Open source Pocket client
Hi, I remember us talking about an open source Pocket client, but it has occurred to me: isn't the Firefox built-in client open-source? I know Firefox on Android isn't entirely open source (unless you use Fennec from F-droid like me), but I thought it was on desktop. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-03-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Next Meeting - Tonight at 8 pm
On 02/02/2021 12:18, Terry Coles wrote: > All, > > The next Online Meeting is tonight at 8 pm using Jitsi. > > Simply click on the following link and you will be taken to the Meeting > using your default browser: > > https://meet.jit.si/dorset-lug > > Chrome or Chromium are probably better than Firefox for using Jitsi. An > alternative to installing one of those two is to obtain it bundled especially > for Jitsi from: > > https://github.com/jitsi/jitsi-meet-electron/releases > > This should be as simple as > > wget -q https://github.com/jitsi/jitsi-meet-electron/releases/download/ > v2.4.2/jitsi-meet-x86_64.AppImage > chmod +x jitsi-meet-x86_64.AppImage > ./jitsi-meet-x86_64.AppImage > > and then entering ‘dorset-lug’ as the meeting ID. > > Hope to see you all this evening. I'll be there but will probably be a bit late as I am talking to my tutor for the dissertation module at 7:30. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Problem using Chromium to log in to Web Page Secured with flask-httpauth
On 25/01/2021 12:34, Terry Coles wrote: > Hi, > > I have set up basic protection for my Minster Control Web page using the > information in the man page for flask-httpauth see: > > http://manpages.ubuntu.com/manpages/groovy/man1/flask-httpauth.1.html > > My App uses the code in the first example given and works fine, except that > if I > log in to my secured page using Chromium, I am never challenged for the > password after the first login, even after I have shut down the browser. If > I > log in with Firefox, everything is good, ie, once logged in, I can re-enter > the page without being challenged again until I shut down the browser. In > that instance I am challenged again when I surf to the protected page. > > Chromium has not saved the password. > > Can anyone explain what is going on and is this likely to be a security issue? I imagine this will be to do with the cookie and session cookie settings you have set for those browsers. My guess is you have Firefox set to clear them when closed, but not with Chromium. Whether this is an issue depends on how you judge the situation to be, but also how long it takes for the login cookie to expire. The longer it takes for it to expire, the bigger the risk of someone getting access if they eg pocket someone's device. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Flask web development book
On 23/01/2021 18:22, Keith Edmunds wrote: > It's all available online (I think it's close to the same as the book): > http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world > > And it's excellent. Looks pretty similar, if a bit less complete. Probably saved me £20 though, definitely very useful, so thanks :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Old Books - Junk?
On 06/01/2021 10:21, PeterMerchant wrote: > If anybody wants them, I will keep them aside, otherwise it's > recycling bin or charity shop. > > Computer Security - Gollman second Edition 2006 (3rd edition came out > in 2011) > > Local Area Network Management, Design and Security - Mikalsen 2002 > > Principles of Network and System Administration - Burgess 2000 (2nd > Edition came out in 2004) > > What should I do with them? > > [ don't say put them back on my shelves for another 10 years!] > > Peter M. > > Okay, I've finally had a look, hopefully I didn't take too long. Computer Security - Gollman looks interesting to me, would you want anything for it? The other two are actually still for sale - you might be able to eBay/similar them and get some money back. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Flask web development book
Hi there, I was wondering if any of you have bought this and can comment on whether it's any good - book at https://www.barnesandnoble.com/w/flask-web-development-miguel-grinberg/1118175139. It can also be had off of ebay for much less than this. The Flask documentation is great, but a few similar books for other things like Java and wxPython have been very helpful for me, and it's nice to read a paper book and get off of screens, so any comments would be great :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Using Flask to render a set of Buttons.
On 22/01/2021 12:51, Terry Coles wrote: > On Thursday, 21 January 2021 14:28:42 GMT Ralph Corderoy wrote: >> https://stackoverflow.com/questions/19794695/flask-python-buttons/27952991#2 >> 7952991 is the same approach as the book. You may find the book better >> written than trying to piece it together from SO answers. :-) > Ralph, > > You are right of course about the dangers of piecing a solution together from > SO answers. It took me quite a while but I was able to get something working > using the preferred solution at that site. > > The trouble with Stack Overflow and other like sites is that they often > assume > that the reader already knows everything about the subject, except for the > problem under discussion. In this case a novice like me had to 'know' that > the 'request' method had to be explicitly imported from the flask module. I > had the same problem with 'form'. > > That solution was provided by Miguel Grinberg the author of the Tutorial that > you suggested that I use instead, so I was fairly comfortable that I would > get > something to work. > > On a related topic I bought Miguel's book from Amazon, but found that the > Kindle edition was not searchable which made it extremely hard to find things > without reading the book from cover to cover (416 pages, no index, no Table > of > Contents). Not good for a text book. > > Eventually I discovered that indexing worked in that book if I viewed it > using > the Kindle Reader in Firefox, but not when using the Kindle Reader in > Chromium. Also the book is not searchable on my Kindle device. I'm glad you found a solution. Is it on gitlab? Might be useful for me to see it as well, seeing as I'll be using Flask soon for the visitor GUI. I didn't even know ebooks could be viewed in browsers, that's good! Perhaps filing a bug for Firefox would be a good idea? I'm sure other people are affected by this issue too. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] CentOS petition
On 21/01/2021 12:59, Tim Waugh wrote: > > Here's the latest about this: > > https://www.redhat.com/en/blog/new-year-new-red-hat-enterprise-linux-programs-easier-ways-access-rhel > > Tim. > */ Oh that looks really good, thank you for sharing :) Makes me feel quite a lot better about the whole thing. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Using Flask to render a set of Buttons.
On 21/01/2021 13:03, Terry Coles wrote: > On Thursday, 21 January 2021 12:52:44 GMT Hamish McIntyre-Bhatty wrote: >> I haven't looked at Flask in detail, but one way to do it would be to >> use a GET or POST value when the button is submitted in the form, so it >> eg redirects to /control.html?action=play. > Thanks Hamish. I did wonder if that might be the only way, but it seemed a > bit clunky and felt that there ought to be something more sophisticated that > I've missed. > > One trouble that I've found with Flask is that on its own its fairly > straightforward, but much of what is done is done my the loading of flask > modules and other helper libraries. It isn't always obvious which of those > module or libraries to study to find what I need. > > That's why I've been trying to find an example from someone who has done > something similar. I've found a number of examples using javascript (I even > used one myself for the original WMT Webserver), but they all simply redirect > to another page. I need to execute a line of Python. The only other way I particularly know is to use client-side Javascript to make that request without reloading the page. I know it is also possible to respond to the result from the server, for example with a callback, but I can't provide a good example of how to do this because I lack the expertise. If I spend some time later I my be able to find you an example of it. The only way (that I know of) to do this is to send a request to the server in some way to trigger it to execute the Python line, eg by redirecting to a page or sending a request through some other means. I'm quite happy to be proven wrong though, I'm sure there must be a proper web developer on here who we can both learn from. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Using Flask to render a set of Buttons.
On 21/01/2021 12:39, Terry Coles wrote:
> Hi,
>
> I'm building a Web page to allow users to monitor key status information on
> one of the Pis at WMT. I also want to create a page where users can press a
> button to carry out certain tasks, such as Start or Stop some music playing
> or
> select a new Playlist. The Player is written in Python. Altogether, there are
> around 10 or more control functions that I want to invoke.
>
> I have the monitoring page working reasonably well using a suitable html
> template and a Flask route which populates the status values on the rendered
> page. However, the control side is making me think a bit. I've found quite a
> lot of tutorials and text book examples that use forms and a 'Submit' button,
> but so far nothing quite like what I want.
>
> What I need is for the user to be presented with set of buttons that, when
> pressed, cause a bit of Python code to be executed that communicates with the
> Player program. On receipt of the message, the Player program will trigger a
> set of functions to carry out the requested action. The communication side
> is
> sorted; all I need is an example or a link to a suitable tutorial that
> illustrates the Flask code that I need to make it happen and if necessary,
> the
> associated template.
>
> Here is a bit of pseudo code to illustrate what I'm trying to do:
>
> @app.route('/control')
> def control():
> Add a Button called "Stop the Music"
> ON click run:
>socket.write("Stop the Music")
>
>Add another button...
>
>etc.
>
> Can anyone help?
I haven't looked at Flask in detail, but one way to do it would be to
use a GET or POST value when the button is submitted in the form, so it
eg redirects to /control.html?action=play.
Then, the server side-script can examine the value of the action
attribute when the page reloads, and perform the action. It could also
render a little confirmation message to say whether or not it was
successful.
If you want to avoid page reloads, you'll likely need some JavaScript to
catch the form submission and send the request to the server through
something like the XHR facility.
Might need a little more info on exactly how you want it to behave though.
Unfortunately I don't have the knowledge to help much beyond
conceptualising this at the moment, but I'm sure someone here does web
development as a profession/has some time to find something for you.
Hamish
signature.asc
Description: OpenPGP digital signature
--
Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00
Check to whom you are replying
Meetings, mailing list, IRC, ... http://dorset.lug.org.uk
New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Raspberry Pi Pico
On 21/01/2021 09:50, Terry Coles wrote: > On Thursday, 21 January 2021 09:25:02 GMT Hamish McIntyre-Bhatty wrote: >> This is a very interesting product. That said, I agree with you Terry in >> that I think it would have been hard to even get Ethernet to work with >> something like this, let alone talking to a MySQL database - probably >> unsuitable for the River System, or at least the direction it went in >> given the hardware we had at the time. > At the time, there was no thought of networking the devices; this was when I > did the lighting controller. This would have been perfect for that because > in > the end I was unable to get PWM to work without flickering using Python > because > the GPIO Library didn't (doesn't?) support the hardware PWM on Pin 18. I > got > it working with WiringPi and C, although the C code is hardly rocket science > with only around 160 lines of code around a third of which are comments. > > The train controllers would have been a perfect choice for this , but I > couldn't have used it for any of the four audio players on the site. I agree > with you that we certainly would have struggled to develop the software for > the River System in its present form. > > From a hardware standpoint we would have been fine for any of the River > System > devices if USB networking could be made to work. If I were starting today, > I > would seriously consider using a Pi Zero at each location to do the grunt > work, connected to the Pico's using USB Networking, like I've just done on > the > new Minster hardware. That's true, if USB is fairly functional that perhaps USB NICs could be made to work. It will still need a TCP/IP stack though. What would the Pico's be doing in that design? All the hardware-related stuff like interfacing with probes? I realise this is very much a hypothetical discussion, but still interested :). Does the USB Raspberry Pi networking work well for the new Minster system? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Raspberry Pi Pico
On 21/01/2021 08:11, Terry Coles wrote: > On Thursday, 21 January 2021 07:59:12 GMT PeterMerchant wrote: >> I notice that it doesn't have WiFi, and I wonder if it could be programmed >> from the Arduino IDE as 'c' is mentioned. Perhaps that will come. I have > Do you not compile the C code before uploading it to an Arduino or similar? > I've never used an Arduino but have always assumed that the compiler was in > the IDE not the device. > >> been using a Wemos D1 which has some of these features plus Wifi, and >> programmed in c. > That's an interesting device too, albeit a lot less capable than this latest > offering. > > The microPython is the attractive thing. When i was evaluating the options > at > WMT in 2016, I plumped for the RPi Zero because it was half the price of the > cheapest Arduino, but more importantly it supported Python. > > Since the WMT is largely maintained by volunteers, I didn't want to use a > language that might be hard to use for novices. Since Python was going up > the > popularity charts even then, and is quite powerful, I felt it was the right > choice. I still do. This is a very interesting product. That said, I agree with you Terry in that I think it would have been hard to even get Ethernet to work with something like this, let alone talking to a MySQL database - probably unsuitable for the River System, or at least the direction it went in given the hardware we had at the time. I must admit I struggle to see the difference between a microcontroller and a very low end SoC/SBC, apart from the analogue inputs (but AFAIK some SoCs/SBCs include analogue inputs too). Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Recommendations for webcam?
Extra: It seemed to work okay for Jitsi though, so that's good (I used by headset for audio input). Hamish On 09/01/2021 22:05, John Horne wrote: > On Fri, 2021-01-08 at 15:27 +0000, Hamish McIntyre-Bhatty wrote: >> Hi, >> >> So as I mentioned I'm going to be buying a webcam as my existing >> solution isn't working. >> >> I imagine some of you probably use dedicated USB webcams, so perhaps you >> could tell me which ones you're using? >> >> I'll do research anyway, but it seems webcams are one of those things >> where Linux support is still poor, so any recommendations are welcome :) >> > Hi, > > I bought my webcam during the first lockdown as I needed it for meetings. At > the time a lot of Logitech webcams were either unavailable or very expensive - > I didn't need anything fancy, just something that worked and gave a reasonable > image and audio. > > I bought a papalook webcam from Amazon > (https://www.amazon.co.uk/gp/product/B07HF56LX8/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8=1) > > It works fine out of the box with Fedora 32 and 33 with Zoom. For use with > Microsoft Teams (used by my employers) required one minor config change to get > the audio to work (had to add my local account to the 'pulse' group I think). > > Using online tests of the webcam indicated that it gave a good image, and the > audio was okay (it was clear enough). My colleagues confirmed that both of > these were fine from their end. > > Overall I'm very happy with the webcam. > > If you're a little bit security paranoid like me, I also bought some webcam > lens covers too > (https://www.amazon.co.uk/gp/product/B075VYMPHW/ref=ppx_yo_dt_b_asin_title_o06_s00?ie=UTF8=1) > These are trivial to fit and work fine. > > > > John. > > -- > John Horne | Senior Operations Analyst | Technology and Information Services > University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK > > [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> > > This email and any files with it are confidential and intended solely for the > use of the recipient to whom it is addressed. If you are not the intended > recipient then copying, distribution or other use of the information > contained is strictly prohibited and you should not rely on it. If you have > received this email in error please let the sender know immediately and > delete it from your system(s). Internet emails are not necessarily secure. > While we take every care, University of Plymouth accepts no responsibility > for viruses and it is your responsibility to scan emails and their > attachments. University of Plymouth does not accept responsibility for any > changes made after it was sent. Nothing in this email or its attachments > constitutes an order for goods or services unless accompanied by an official > order form. signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Recommendations for webcam?
Thanks for all the suggestions. So in the end, I got this generic one: https://smile.amazon.co.uk/gp/product/B0899RJQ95/ref=ppx_yo_dt_b_asin_title_o02_s00?ie=UTF8=1 This was mostly because of cost. It can do 1080P, but (and this is a big but) it has issues depending on the mode it's used in. It can only do 1080P30FPS in M-JPEG mode, without audio. If you use YUYV the framerate is poor unless you use VGA mode. I have not tried the built in mic. So unless like me you don't plan to use the mic, I wouldn't recommend this. It is very cheap though. Here's the output from v4l2-ctl --list-formats-ext: ioctl: VIDIOC_ENUM_FMT Type: Video Capture [0]: 'MJPG' (Motion-JPEG, compressed) Size: Discrete 1920x1080 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1440x1080 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x1024 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x720 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 320x240 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) [1]: 'YUYV' (YUYV 4:2:2) Size: Discrete 1920x1080 Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1440x1080 Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x1024 Interval: Discrete 0.200s (5.000 fps) Size: Discrete 1280x720 Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Size: Discrete 320x240 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.200s (5.000 fps) Hamish On 09/01/2021 22:05, John Horne wrote: > On Fri, 2021-01-08 at 15:27 +0000, Hamish McIntyre-Bhatty wrote: >> Hi, >> >> So as I mentioned I'm going to be buying a webcam as my existing >> solution isn't working. >> >> I imagine some of you probably use dedicated USB webcams, so perhaps you >> could tell me which ones you're using? >> >> I'll do research anyway, but it seems webcams are one of those things >> where Linux support is still poor, so any recommendations are welcome :) >> > Hi, > > I bought my webcam during the first lockdown as I needed it for meetings. At > the time a lot of Logitech webcams were either unavailable or very expensive - > I didn't need anything fancy, just something that worked and gave a reasonable > image and audio. > > I bought a papalook webcam from Amazon > (https://www.amazon.co.uk/gp/product/B07HF56LX8/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8=1) > > It works fine out of the box with Fedora 32 and 33 with Zoom. For use with > Microsoft Teams (used by my employers) required one minor config change to get > the audio to work (had to add my local account to the 'pulse' group I think). > > Using online tests of the webcam indicated that it gave a good image, and the > audio was okay (it was clear enough). My colleagues confirmed that both of > these were fine from their end. > > Overall I'm very happy with the webcam. > > If you're a little bit security paranoid like me, I also bought some webcam > lens covers too > (https://www.amazon.co.uk/gp/product/B075VYMPHW/ref=ppx_yo_dt_b_asin_title_o06_s00?ie=UTF8=1) > These are trivial to fit and work fine. > > > > John. > > -- > John Horne | Senior Operations Analyst | Technology and Information Services > University of Plymouth | Drake Circus | Plymouth | Devon | PL4 8AA | UK > > [http://www.plymouth.ac.uk/images/email_footer.gif]<http://www.plymouth.ac.uk/worldclass> > > This email and any files with it are confidential and intended solely for the > use of the recipient to whom it is addressed. If you are not the intended > recipient then copying, distribution or other use of the information > contained is strictly prohibited and you should not rely on it. If you have > received this email in error ple
Re: [Dorset] Am I allowed a moan?
On 18/01/2021 17:24, Tim wrote: > > On 17/01/2021 14:40, PeterMerchant wrote: >> I have been trying to work with a group 'Wimborne laptops for >> schools' for children at home who do not have computers to work with. >> A friend gave me her old computer to donate and I had my own old one. >> I was told that the sufficient requirement was for a browser. So I >> installed Xubuntu Linux on these, and updated them to the latest >> Firefox. These are 32 bit machines circa 2005. >> >> So I fixed them up with webcams [Aside: Cheese isn't the best tool >> for evaluating webcams, use vlc] and donated them. They were set up >> identically for username and auto-login. >> >> Meanwhile I phoned Q.E School and they said that they only wanted >> machines post 2016 that they could log into and put their own >> software and updates on. St. Michaels Middle School gace examples >> of programs that they wanted to run over the browser and they were ok. >> >> Then I get this message back from the group: Hi Peter Just in case >> you get any more laptops schools are unable to use the Xubuntu >> Operating system. it has to be windows 10 pro just so you don't waste >> your time . >> >> Which is a bit stupid as quite a few kids are using parents mobile >> phones and ipads - Not W10! >> >> So they want to scrap my laptops and use them for spares - Sure!!! >> >> Rant over. If I get them back I will take them to a local school myself. >> >> PeterM >> >> > > Beggars with the golden begging bowl springs to mind > > Tim H Mm, this seems pretty ridiculous, especially since that's not what they told you. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-02-02 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Laptop cooling
On 08/01/2021 16:06, Terry Coles wrote: > On Friday, 8 January 2021 15:26:33 GMT Hamish McIntyre-Bhatty wrote: >> In case anyone's interested, replacing the heatpipe for my laptop did >> fix the issue, it runs much cooler and can now continually boost even >> when under full load for several minutes, rather than thermal throttling. > I wasn't aware that you had a cooling issue on your laptop; perhaps you > raised > it after I went to bed on Tuesday? > > Anyway, years ago I ran a project to develop a military test set based on a > rugged laptop. We replaced the heat pipes in that with longer ones and one > of > the suppliers explained how they work. Apparently it is not uncommon for the > pipe to leak allowing the partial vacuum to fill and the vaporisation process > to stop. > > Just in case you didn't know :-) Thanks, that's useful. I thought heatpipe failure was extremely uncommon, but perhaps I was mistaken. Good to know anyway :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Laptop cooling
In case anyone's interested, replacing the heatpipe for my laptop did fix the issue, it runs much cooler and can now continually boost even when under full load for several minutes, rather than thermal throttling. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Old Books - Junk?
On 06/01/2021 10:21, PeterMerchant wrote: > If anybody wants them, I will keep them aside, otherwise it's > recycling bin or charity shop. > > Computer Security - Gollman second Edition 2006 (3rd edition came out > in 2011) > > Local Area Network Management, Design and Security - Mikalsen 2002 > > Principles of Network and System Administration - Burgess 2000 (2nd > Edition came out in 2004) > > What should I do with them? > > [ don't say put them back on my shelves for another 10 years!] > > Peter M. I'll look these up, but it sounds as though I might be interested in at least one of them. I'll get back to you soon. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Intel RST?
On 05/01/2021 14:54, Tim Waugh wrote: > Hi, > > I have a laptop with a spare 2.5" drive bay I'd like to install a new disk > into (and run Linux on). > > The laptop uses Intel Rapid Storage Technology, which seems to be some sort > of RAID controller. I can switch it to regular AHCI in the BIOS, but if I > do that I lose the ability to dual boot with the pre-installed OS. > > Has anyone come across this before? Is there a way to stick with Intel RST > but have un-RAIDed disks installed? > > Alternatively, is there a way to boot into the (Windows) installation on > the already-RST-formatted disk, after switching the BIOS to AHCI? Obviously > I could switch the BIOS back to RST if need be for booting on that, but can > that be avoided? > > Tim. > */ I think I had this problem on my old desktop PC. The problem is that, while Windows can be migrated (it would consider this a drive migration), it apparently won't load the drivers for SATA disks unless it sees a SATA disk when powered on. So you'd need to find a way to have the extra drive plugged in, and in SATA, and boot Windows once, before switching to AHCI mode. I'm not sure if that's possible. Otherwise I guess you could reinstall Windows? That's kind of a big pain though. From what I recall, Linux doesn't play well with Intel RST, and doesn't recognise the drives, but that may have since changed. Hope this helps, Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Migrating from VirtualBox to KVM/QEMU with virt-manager
On 17/12/2020 21:55, PeterMerchant wrote: > On 17/12/2020 16:34, Hamish McIntyre-Bhatty wrote: >> Hi all, >> >> I've had enough of VirtualBox's glitchiness, so I'm going to move as >> many of my VMs to KVM/QEMU with virt-manager as possible. This should >> also be more performant, or so I hear. >> >> I've done this before with Linux VMs and it's fairly easy, but I was >> wondering if anyone has heard of a way to move an activated Windows >> install without deactivating it? Mine is an OEM copy, so I'll have to >> pay for it again if it deactivates. >> >> I could of course just leave that one alone, but it'd be good to get >> them all working in KVM. I sadly have to keep VirtualBox around for a >> few things, but might as well minimise those. >> >> Hamish >> >> > Admin, the next meeting date in the footer needs correcting. > > PM. NB: I did find a way of doing this, if anyone's interested. Of course, I'm not suggesting any do this to avoid paying the license fee when they haven't already paid for the operating system, rather to virtualise an existing Windows install or move between VM hypervisors on the same hardware. The links: https://support.microsoft.com/en-us/windows/reactivating-windows-10-after-a-hardware-change-2c0e962a-f04c-145b-6ead-fb3fc72b6665 and: https://gist.github.com/Informatic/49bd034d43e054bd1d8d4fec38c305ec Provide some useful information for doing this. In my experience you still have to reinstall the OS, rather than just cloning, but hopefully this is still useful to someone. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] CentOS petition
Hi there, I don't know if any of you use/target CentOS/RHEL, but if you do you may be interested in this petition: http://chng.it/8HWPR6j7JN I am doubtful as to the effectiveness of such a thing, but I have gone ahead and signed it anyway. I think Red Hat is making a big mess for their users (and itself) through these plans for CentOS. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2021-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] No Response in alsamixer and Error from amixer
On 18/12/2020 12:41, Terry Coles wrote: > Hi, > > Some of you are aware that I've been re-engineering the Minster Bells and > Music Player at Wimborne Mode Town. Amongst other things I split the > functionality into two Pis (the original Pi 3 and a Pi Zero); both are > equipped with an Adafruit Stereo 3 W Amplifier Bonnet: > > https://thepihut.com/products/adafruit-i2s-3w-stereo-speaker-bonnet-for-raspberry-pi > > The Pi 3 takes care of the music that plays in the Nave and I configured that > first and using a cut-down version of the original code, it has been running > satisfactorily for a week or two. > > I then backed up its SD Card and created a new one using the Music Player > image as a starting point. Again, using a cut-down version of the original > code installed on the pi Zero, I have now got the bells to ring (almost) > perfectly. > > The one problem that I have is the volume setting command. In both programs > I > have a line: > > subprocess.call(['amixer', '-c', '0', 'set', 'PCM', 'playback', nave_vol]) > > In the music program this works perfectly every time, but in the bells > program > it fails intermittently with: > > amixer: Invalid command! > > After this failure alsamixer has no control over the volume level. However, > once the amixer command is working so is alsamixer. > > Any ideas on what I should check? AFAIK there is no difference between the > two > devices other than the underlying Pi type. I've tried all sorts of > combinations of amixer command from the man page, but when it works it works > and when it doesn't it seems to refuse to do anything. Quick thought: Print out the exact commandline of: ['amixer', '-c', '0', 'set', 'PCM', 'playback', nave_vol] Every time just in case some previous error or something is getting into those variables? You've probably already checked that the commandline is the same when it stops working I guess, in which case this isn't very useful, but it's the first thing I'd try. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Migrating from VirtualBox to KVM/QEMU with virt-manager
On 17/12/2020 16:38, Keith Edmunds wrote: > On Thu, 17 Dec 2020 16:34:22 +, hamis...@live.co.uk said: > >> a way to move an activated Windows >> install without deactivating it? Mine is an OEM copy > You cannot (officially, at least) move OEM Windows to any other device. Yeah, I know, but it'll be running on the same bare metal, and I paid for it, so I think it's a reasonable thing to ask in this instance. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Migrating from VirtualBox to KVM/QEMU with virt-manager
Hi all, I've had enough of VirtualBox's glitchiness, so I'm going to move as many of my VMs to KVM/QEMU with virt-manager as possible. This should also be more performant, or so I hear. I've done this before with Linux VMs and it's fairly easy, but I was wondering if anyone has heard of a way to move an activated Windows install without deactivating it? Mine is an OEM copy, so I'll have to pay for it again if it deactivates. I could of course just leave that one alone, but it'd be good to get them all working in KVM. I sadly have to keep VirtualBox around for a few things, but might as well minimise those. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Distributed computing for COVID
On 14/12/2020 10:15, Tim Waugh wrote: > On Mon, 14 Dec 2020 at 09:50, Hamish McIntyre-Bhatty > mailto:hamis...@live.co.uk>> wrote: > > I was wondering, now that there are multiple vaccines in > development/in > use, whether we still think it's worth doing distributed computing for > COVID research. > > > I plan to continue for the moment at least. Even though the aim is to > find something new, which must then go through lengthy safety trials > etc, along the way some other insight might suggest a faster approach > using something pre-existing. In any case, at this point I have no > idea when vaccination might roll out to my age group; it's still > possible it would be after a safety trial of some new discovery. > > There are lots of things no-one knows about any of the vaccines, > notably how long they will last, or how long that time compares with > the time it would take to roll out a refresh. > > Tim. > */ That is very fair. I shan't stop completely then, and will keep it going on the Pi. The fan noise of the laptop 24/7 and increased fan noise on the desktop is getting a bit boring after several months XD. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Distributed computing for COVID
Hi all, I was wondering, now that there are multiple vaccines in development/in use, whether we still think it's worth doing distributed computing for COVID research. Currently I have an old laptop and a Pi 3 doing this 24/7, and my desktop does it when it's on while I work. I'm thinking I might scale back to just using the Pi since it's always on anyway. Thoughts? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-01-05 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Paths relative to a current working directory whose path contains symbolic links
On 09/11/2020 17:45, Patrick Wigmore wrote: > Thank you so much for that informative response, Ralph. > > So much interesting history! > > On Mon, 09 Nov 2020 15:00:55 +, Ralph Corderoy wrote: >> The program can't know which. It shouldn't try and >> guess but instead just pass the argument to open(2), etc. > That was pretty much my conclusion. It was interesting to look for a > way to do it, but I didn't really like the idea of writing a long- > winded workaround that bypassed well-trodden standard functions. > > If programs can't resolve relative paths in the same way that the > shell does, then I might have expected the shell to turn the relative > paths into absolute ones before passing them as arguments to programs. > > But that opens up a can of worms too. The shell (probably) can't know > whether I intend a string that looks like a relative path to be > treated as a reference to a file, or as a literal string that just > happens to look like a path, or as a path relative to something else > that the program is going to know about! > >> If that bites the user, e.g. by trampling the wrong file, then the >> user will have learnt symlinks are a bad idea and alternatives >> should be sought where possible. :-) > I have symbolic links to give the effect of selectively placing some > of the subdirectories of my home directory on an HDD. The rest of the > home directory is on an SSD. It has worked "well enough" for quite a > while now, but I suppose bind mounts might be a better idea. > > > Patrick Interesting to know about all this. I do a similar thing to you Patrick, but for file syncing with Nextcloud over my home network. Unfortunately I'm not sure bind mounting would work for me in this case, but if you have a go and it works, please let me know :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-12-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
Just noting that with the latest system and virtualbox updates that this problem has gone away. I have no idea why it ever happened, but virtualbox is now stable, and the kernel memory is now only 300MB at the end of the day instead of 12GB. It's still more than is ideal I guess, but it no longer is a big problem. Just as well seeing as I have other problems to fix now XD Hamish On 10/10/2020 12:48, Ralph Corderoy wrote: > Hi Hamish, > >> I have found that once logged in, the SUnreclaim value only goes up >> when I'm looking at the X server output. If I switch to a virtual >> terminal, it (almost) stops climbing. > Create a new user. Do nothing to their account. Ensure you're not > logged in. Log in as them and see if SUnreclaim climbs whilst using > their X server. This will help tell if it's something you've done to > your X session, e.g. a constantly adjusting background image showing the > phases of the moon. > >> Display driver related issue maybe? > Yes, maybe. > >> I'm not quite sure where to report it, so I guess I'll ask on the Mint >> forums first and see if anyone else has experienced this. Sound like a >> good starting point? > Yes. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-11-03 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 16/09/2020 12:24, Hamish McIntyre-Bhatty wrote: > On 16/09/2020 12:19, Hamish McIntyre-Bhatty wrote: >> On 11/09/2020 21:49, Hamish McIntyre-Bhatty wrote: >>> On 11/09/2020 10:04, Ralph Corderoy wrote: >>>> >>> I'm gonna have to shut this system down, but in the meantime I'm open to >>> any more suggestions. I'll try a bunch of different things I guess, and >>> see f it makes any difference. May also see if it happens on a live disk. >>> >>> Hamish >> Turns out this seems to happen (or at least be much worse) when I'm >> using encrypted swap space with LUKS. I shall report this to the Mint team. >> >> It doesn't happen when live-booted from a USB stick on the same system, >> so good news is that it's probably something with my configuration like >> the above, rather than a bit of a disaster for Mint 20 in general. >> >> For the NAS box, I'm going to cross compile some more tools. On my list >> are top and free (better versions), and htop. Any ideas for small tools >> that might be useful? Bearing in mind the ARMv5 architecture and the age >> of the platform (Linux 2.6, glibc 2.8, GCC 4.7) I might not be able to >> compile big things, but I have had luck with many things. >> >> GCC 4.7 is the newest thing I could get to work, but it seems to compile >> most things without a problem even now. >> >> Hamish > Nope, I was wrong, it's something else. I'll figure it out eventually, > but I'm mostly just glad it doesn't affect vanilla mint. Fortunately I > have an overkill amount of RAM so it doesn't matter anyway. At least > it's probably not hardware - also occurred on my old Intel system w/ > NVIDIA GPU, but now I'm all AMD. > > Hamish Using the script Ralph sent me, and a clone of my installation to an external HDD, I have found that once logged in, the SUnreclaim value only goes up when I'm looking at the X server output. If I switch to a virtual terminal, it (almost) stops climbing. Display driver related issue maybe? I'm using a Radeon RX460 with the open source AMDGPU driver. I'm not quite sure where to report it, so I guess I'll ask on the Mint forums first and see if anyone else has experienced this. Sound like a good starting point? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-11-03 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] GPL X server VcxSrv?
Hello, I was wondering if any of you have come across this: https://sourceforge.net/projects/vcxsrv/ It seems to be a reliable, well-respected X server port for Windows. I'm thinking of bundling it with my program. However, the Xorg X server is released under a variety of licenses (MIT + others), so I was wondering if anyone can shed some light on how this is GPL and whether that is breaking license terms? As an aside, there's also X410 (https://x410.dev/), which is actually a non-free X server, and how that isn't breaking license terms. Am I being a bit dumb? :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-11-03 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Windows-based solar panel-related software
Hi Terry, I believe you said during the call that you keep a Windows VM around just to occasionally run a piece of software that talks to your solar panel energy output monitor/similar? I'm sure you probably looked into this, and I think you said you don't use it much, but have you tried running it in WINE? Hamisb signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-11-03 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Next Online Meeting - Tonight at 8pm
Excellent, I'm looking forward to boring you all to death with Cygwin stuff :) Hamish On 06/10/2020 12:35, Terry Coles wrote: > All, > > The next Online Meeting is tonight at 8 pm using Jitsi. > > Simply click on the following link and you will be taken to the Meeting > using your default browser: > > https://meet.jit.si/dorset-lug > > Chrome or Chromium are probably better than Firefox for using Jitsi. An > alternative to installing one of those two is to obtain it bundled especially > for Jitsi from: > > https://github.com/jitsi/jitsi-meet-electron/releases > > This should be as simple as > > wget -q https://github.com/jitsi/jitsi-meet-electron/releases/download/ > v2.4.1/jitsi-meet-x86_64.AppImage > chmod +x jitsi-meet-x86_64.AppImage > ./jitsi-meet-x86_64.AppImage > > and then entering ‘dorset-lug’ as the meeting ID. > > Hope to see you all this evening. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Interesting Linux and old tech Youtube channel
On the off chance some of you are interested, I found this channel that goes in to some detail about Linux and getting things working on obscure and old architectures. The guy is kinda opinionated and grumpy, but the content is quite good (and gloriously nerdy) if you're in the mood so I thought it might be worth a share. https://www.youtube.com/user/renerebe Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] How to Cancel Existing systemd.timer(5) OnCalendar Values.
On 09/09/2020 12:15, Keith Edmunds wrote: > On Wed, 9 Sep 2020 09:44:44 +0100, hamis...@live.co.uk said: > >> so has anyone experienced cron jobs sporadically (kind of randomly) not >> running when they're meant to? > No, never (and I've been using Linux since the early 90s, and UNIX before > that). > > Set up your cron jobs so that the command is: > > [...] your-command 2>&1 | /usr/bin/logger -t 'my cronjob' > > Then take a look in your syslog files after the cron job has apparently > not run. > > Linux Tips: https://www.tiger-computing.co.uk/category/techtips/ Turns out cron is fine on the pi, the problem is my system time not syncing for some unknown reason. The joys of debugging... Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Loss of OS during upgrade
On 25/09/2020 10:58, C Wills wrote: > Hi All > Update on my problems when upgrading Mint 19.3 to Mint 20. > > Thank you all for your help, I tried most suggestions and finally, > yesterday, got everything working. > > Ralph: Thank you for all our suggestions and I tried to follow the > last set, and failed (later found I'd made a spelling mistake!!). > > I did manage to recover all my data files to the separate disk but did > not need them in the end (now stored safely, just in case). > > After many attempts to upgrade I've come to the conclusion that the > upgrade DVD is faulty, as that always failed. > During one recovery attempt (back to 19.3) it repartitioned the SDA > disc and added a EFI partition, as well as the OS (19.3). It replaced > sda1 & 2 with the efi (fat32) and OS partition(ext4), it deleted the > Swop but did not touch sda3 or 4. The efi/OD partitions ended up at > sad5&6, no sda2 or swop. > Don't understand why or how but it all works! > > I'll have to wait till I can get a 'proper' Mint 20 disc before trying > again. Must have tried 6 or 7 times to upgrade hence the thinking > it's a DVD copy problem as at no time did it get passed the loss of > efi problem; even after having a separate efi/fat32 partition. > Thanks again for all the help, it kept me trying! :-) Glad you got it back in working condition. I would suggest you try a package based upgrade - this is what they announced on linuxmint.com as working, as it has worked on three different computers for me. That might yield more success. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 12/09/2020 14:07, Ralph Corderoy wrote: > Hi Hamish, > >> Any thoughts about diagnosing storage unmounting with no explanation >> is dmesg? This seems rather strange to me. I'm using a powered hub >> too now, in case I forgot to mention. Other volumes on the same >> external disk stay mounted. > - How does the filesystem which keeps unmounting differ from the other > ones on that disk which don't? This one is in a LUKS container. The others are not. > - What's that filesystem used for? File syncing between my laptop and desktop with Nextcloud. > - Does ‘sudo lsof /dev/sda1’, adjusted as appropriate, show surprising > programs using that filesystem? Will get back to you on that in a few days :) > - Is it possible it is deliberately unmounted from userspace due to some > forgotten or erroneous program? > - If the mount-point is altered, does the problem continue? > This may be impractical to test. Possible I guess. I will give this a try if it fails again. > - Have you fun a *forced* fsck on the unmounted filesystem? Yes, it comes up okay. > - Have you read every byte of the block device? > - Does putting it under stress in this manner make an unmount more > likely? Yes, it seems okay as far as badblocks' read only test says. Previously yes, but with the new USB driver is seems more stable. It still did crash after adding the new driver though. I'm pretty sure the drive has enough power - it ran mostly fine even without the hub, but it now has an extra 2A available from a generic but known good power supply - should be good. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 16/09/2020 12:19, Hamish McIntyre-Bhatty wrote: > On 11/09/2020 21:49, Hamish McIntyre-Bhatty wrote: >> On 11/09/2020 10:04, Ralph Corderoy wrote: >>> >> I'm gonna have to shut this system down, but in the meantime I'm open to >> any more suggestions. I'll try a bunch of different things I guess, and >> see f it makes any difference. May also see if it happens on a live disk. >> >> Hamish > Turns out this seems to happen (or at least be much worse) when I'm > using encrypted swap space with LUKS. I shall report this to the Mint team. > > It doesn't happen when live-booted from a USB stick on the same system, > so good news is that it's probably something with my configuration like > the above, rather than a bit of a disaster for Mint 20 in general. > > For the NAS box, I'm going to cross compile some more tools. On my list > are top and free (better versions), and htop. Any ideas for small tools > that might be useful? Bearing in mind the ARMv5 architecture and the age > of the platform (Linux 2.6, glibc 2.8, GCC 4.7) I might not be able to > compile big things, but I have had luck with many things. > > GCC 4.7 is the newest thing I could get to work, but it seems to compile > most things without a problem even now. > > Hamish Nope, I was wrong, it's something else. I'll figure it out eventually, but I'm mostly just glad it doesn't affect vanilla mint. Fortunately I have an overkill amount of RAM so it doesn't matter anyway. At least it's probably not hardware - also occurred on my old Intel system w/ NVIDIA GPU, but now I'm all AMD. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 11/09/2020 21:49, Hamish McIntyre-Bhatty wrote: > On 11/09/2020 10:04, Ralph Corderoy wrote: >> > I'm gonna have to shut this system down, but in the meantime I'm open to > any more suggestions. I'll try a bunch of different things I guess, and > see f it makes any difference. May also see if it happens on a live disk. > > Hamish Turns out this seems to happen (or at least be much worse) when I'm using encrypted swap space with LUKS. I shall report this to the Mint team. It doesn't happen when live-booted from a USB stick on the same system, so good news is that it's probably something with my configuration like the above, rather than a bit of a disaster for Mint 20 in general. For the NAS box, I'm going to cross compile some more tools. On my list are top and free (better versions), and htop. Any ideas for small tools that might be useful? Bearing in mind the ARMv5 architecture and the age of the platform (Linux 2.6, glibc 2.8, GCC 4.7) I might not be able to compile big things, but I have had luck with many things. GCC 4.7 is the newest thing I could get to work, but it seems to compile most things without a problem even now. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] How to Cancel Existing systemd.timer(5) OnCalendar Values.
On 13/09/2020 12:08, Ralph Corderoy wrote: > Hi, > > I wrote: >> Now I just have to work out why updatedb.conf(5)'s PRUNE_BIND_MOUNTS >> doesn't seem to have an effect. :-) > Because it's buggy and doesn't work. It's been known about for many > years and not fixed or documented. :-( > > https://bugzilla.redhat.com/show_bug.cgi?id=723279 > https://pagure.io/mlocate/issue/23 Aw :/ Out of interest, what do you need that option/feature for? NB: I'll try the suggestions you offered for the issues I'm having, just taking a little break from the debugging things all week :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 11/09/2020 10:04, Ralph Corderoy wrote: > Hi Hamish, > >> NB: On desktop I seem to have a very high number for "SUnreclaim" in >> /proc/meminfo: >> >> MemTotal: 32812004 kB >> MemFree: 8619976 kB >> MemAvailable: 9572924 kB >> Buffers: 61772 kB >> Cached: 1061212 kB >> SwapCached: 1190212 kB > ... >> Slab: 16832040 kB >> SReclaimable: 397472 kB >> SUnreclaim: 16434568 kB > I think SUnreclaim is a key number to monitor over time. > >> What does SUnreclaimable mean? > Slab is the memory used by the kernel's memory allocator. Think of it > as malloc(3) but with knowledge of the type of item that may be > allocated. Some of the slab allocations could be freed if there were > other demands of memory: ‘memory pressure’. That's SReclaimable. > SUnreclaim is the amount allocated to things which must be kept no > matter how high memory pressure goes. > >> This isn't the same on the NAS box, but either way there are two >> problems to debug here and I guess they could be related. > The desktop PC will be easier because you've more tools and more > upstream parties interested in any report. Does the NAS last the day? > Schedule a nightly reboot, or kexec? > https://wiki.archlinux.org/index.php/Kexec > >> Do I have a kernel memory leak? > Probably. > >>> ‘sudo slabtop -osc’ will give a breakdown. > ... >> Okay, that yields: http://ix.io/2x4T >> >> The total is much smaller than the number in /proc/meminfo (just >> verified it hasn't changed drastically). Bizarre. > That is odd. > > Are you using any VM stuff? Any disk filesystems over than ext4, > e.g. ZFS? Nvidia graphics drivers? Is this the machine where a kernel > driver keeps dying? > > Monitor SUnreclaim at a regular time period, e.g. 30 seconds, so you can > see it climbing. You said you were doing a large upload. If it's the > kind which can recover from being stopped and re-started then see if > your monitoring shows a steady climb during upload which stops if you > kill the upload only to restart when you resume the upload. > >> My swap is meant to be for an emergency, not because some leaky code >> in a driver/the kernel/whatever is somehow managing to use 20gb of >> ram. > Swap isn't reserved as an overflow when RAM runs low. Even when there > is plenty of RAM free, the kernel might decide to swap out some memory > which is not backed by another device because it thinks that memory > would be better used by a cache. > > BTW, curl(1) did download something. ;-) > I'm gonna have to shut this system down, but in the meantime I'm open to any more suggestions. I'll try a bunch of different things I guess, and see f it makes any difference. May also see if it happens on a live disk. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 11/09/2020 10:04, Ralph Corderoy wrote: > Hi Hamish, > >> NB: On desktop I seem to have a very high number for "SUnreclaim" in >> /proc/meminfo: >> >> MemTotal: 32812004 kB >> MemFree: 8619976 kB >> MemAvailable: 9572924 kB >> Buffers: 61772 kB >> Cached: 1061212 kB >> SwapCached: 1190212 kB > ... >> Slab: 16832040 kB >> SReclaimable: 397472 kB >> SUnreclaim: 16434568 kB > I think SUnreclaim is a key number to monitor over time. > Good to know. >> What does SUnreclaimable mean? > Slab is the memory used by the kernel's memory allocator. Think of it > as malloc(3) but with knowledge of the type of item that may be > allocated. Some of the slab allocations could be freed if there were > other demands of memory: ‘memory pressure’. That's SReclaimable. > SUnreclaim is the amount allocated to things which must be kept no > matter how high memory pressure goes. > >> This isn't the same on the NAS box, but either way there are two >> problems to debug here and I guess they could be related. > The desktop PC will be easier because you've more tools and more > upstream parties interested in any report. Does the NAS last the day? > Schedule a nightly reboot, or kexec? > https://wiki.archlinux.org/index.php/Kexec Interestingly it seems okay today. Cache has filled up, but memory usage is fairly stable at 110MB, though still no idea what's using it cos apparently not any process. Previously I think it idled at 40MB. Meminfo says "active" on the NAS box is about 50 MB, but no big slab or SUnrclaim here. Perhaps "free" (busybox 1.20.2) is actually reporting virtual memory here. This is something D-Link compiled so I couldn't be surprised if they effed it up somehow, given the quality of the rest of their software and system configuration. That the HDDs >> Do I have a kernel memory leak? > Probably. Also good to know. My friend on totally different hardware (Intel CPU and iGPU) also might be having this problem (but need to confirm it's not cache), so it might not be a hardware-specific thing, or it might be a very generic driver. > >>> ‘sudo slabtop -osc’ will give a breakdown. > ... >> Okay, that yields: http://ix.io/2x4T >> >> The total is much smaller than the number in /proc/meminfo (just >> verified it hasn't changed drastically). Bizarre. > That is odd. > > Are you using any VM stuff? Any disk filesystems over than ext4, > e.g. ZFS? Nvidia graphics drivers? Is this the machine where a kernel > driver keeps dying? No VMs running (but vbox installed). Mounted filesystems are ext4, vfat for /boot/efi, one NTFS (but IIRC this is a userspace driver with FUSE), and ecryptfs for home directory encryption. I also have encrypted swap on a LUKS container. Driver wise I'm using the open source AMD drivers (I have an RX 460). Mint's driver manage confirms I have no proprietary drivers in use (why I chose an AMD GPU, cos the Nvidia ones are a PITA in Linux). No kernel issues as far as I know. As with the pi, ufw is filling up my syslog, so perhaps I'll disable logging here too so I can see more easily. Good idea to stop mounting the NTFS partition to see if that changes anything? > > Monitor SUnreclaim at a regular time period, e.g. 30 seconds, so you can > see it climbing. You said you were doing a large upload. If it's the > kind which can recover from being stopped and re-started then see if > your monitoring shows a steady climb during upload which stops if you > kill the upload only to restart when you resume the upload. Okay, results (every 30 seconds) are below. Before stopping upload (and all associated pCloud processes): SUnreclaim: 19781000 kB SUnreclaim: 19782916 kB SUnreclaim: 19784140 kB SUnreclaim: 19785252 kB SUnreclaim: 19787588 kB SUnreclaim: 19788968 kB SUnreclaim: 19790540 kB SUnreclaim: 19792240 kB SUnreclaim: 19794108 kB SUnreclaim: 19796244 kB SUnreclaim: 19796692 kB SUnreclaim: 19798848 kB SUnreclaim: 19801344 kB After stopping (and ending all pCloud processes): SUnreclaim: 19805016 kB SUnreclaim: 19807172 kB SUnreclaim: 19808868 kB SUnreclaim: 19810428 kB SUnreclaim: 19811948 kB SUnreclaim: 19813980 kB SUnreclaim: 19815524 kB Looks about the same to me. NB: When run w/ quicker interval I can see it hover up and down a little bit, but on the whole increasing. Interestingly RAM usage hasn't increased overnight as much as previous trends made me expect. Since I've had the screens on this morning, memory usage has started to balloon much quicker again... I do have multiple screens, perhaps somehow related to that and/or to the amdgpu driver?//Although I also had this on my old system w/ an Nvidia card. Hmm. >> My swap is meant to be for an emergency, not because some leaky code >> in a driver/the kernel/whatever is somehow managing to use 20gb of >> ram. > Swap isn't reserved as an overflow when RAM runs low. Even when there > is plenty of RAM
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 21:56, Keith Edmunds wrote: > Using swap is not a bad thing in itself (quite the opposite in many cases). > > The actual process of swapping is expensive, though, and you want to > minimise that. > > One way to see whether you have a lot of swapping going on is to run > "vmstat 1" (you don't need to be root). That will produce a new line of > data every second until you stop it (^C). in the middle are two columns > under the label "swap", one sub-labelled "si" and the other "so". They are > respectively "swap in" (ie, read from disk back into memory) and "swap > out" (the other way). If those columns hold mostly zeros, you don't have a > swap problem. > > How does it look on your system? Here's a sample: procs ---memory-- ---swap-- -io -system-- --cpu- r b swpd free buff cache si so bi bo in cs us sy id wa st 3 0 2267312 738416 421568 7325016 0 0 1024 240 7548 10847 18 3 78 1 0 3 0 2267312 739164 421568 7325036 0 0 0 0 6874 10642 19 3 79 0 0 2 0 2267312 739308 421568 7325036 0 0 0 0 6548 10399 18 2 80 0 0 I agree about swapping, but in my case, using 20GB of RAM with almost nothing open (and no change when I close things) is not good behaviour when my use at cold boot is like 2GB. Please note, this is not cache, as reported by "free" and in /proc/meminfo earlier. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 21:17, Keith Edmunds wrote: > On Thu, 10 Sep 2020 21:03:21 +0100, hamis...@live.co.uk said: > >> Any more ideas? This has definitely revealed something interesting, just >> not sure what it is yet/what to do about it. > > You don't need to do anything about it. Well, it keeps going up and up, and artificially loading memory up uses swap, rather than freeing up any memory, so I would beg to differ, unless you can provide me with a reason for this behaviour (and I wouldn't be surprised if there was a good reason, I just don't know what it is!) :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 15:44, Hamish McIntyre-Bhatty wrote: > On 10/09/2020 15:29, Ralph Corderoy wrote: >> ‘sudo slabtop -osc’ will give a breakdown. > Ignore previous message in moderation queue, using ix.io now. > > Okay, that yields: > > http://ix.io/2x4T > > The total is much smaller than the number in /proc/meminfo (just > verified it hasn't changed drastically). Bizarre. > > Hamish Any more ideas? This has definitely revealed something interesting, just not sure what it is yet/what to do about it. A friend who uses the same distribution is reporting similar behaviour, so if I can pin it down I could maybe report this. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 15:29, Ralph Corderoy wrote: > ‘sudo slabtop -osc’ will give a breakdown. Ignore previous message in moderation queue, using ix.io now. Okay, that yields: http://ix.io/2x4T The total is much smaller than the number in /proc/meminfo (just verified it hasn't changed drastically). Bizarre. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 15:07, Ralph Corderoy wrote: > Hi Hamish, > >> This is strange. If I close eg Firefox, the all the (20+!) listings >> for that process go away. > Did you have quite a lot of tabs in that Firefox? It uses multiple > processes these days, roughly number of tabs plus a few. Yeah, I did. My mistake. I saw the PIDs when using htop. > >> What's weird is they all have PIDs, but there aren't that many PIDs >> shown in the process list (or ps aux). > ‘they all have PIDs’: where are they if they're not in the output of > ps(1)? > > What's the output of > > ( > cat /proc/meminfo > ps xauww | sort -sk6,6n -k2,2n > ) | > curl -sSgF 'f:1=<-' ix.io > > when the machine is having problems. Ignore the NAS for now, it's got > too many unknowns. :-) You, uh, could have told me that it would upload that. I was expecting curl to download something. Ah well, nothing confidential in there. the link is http://ix.io/2x4D Makes sense with the NAS, but that's really the priority task because it outright killed the river system software yesterday with no explanation (either in cmdline output or the software's logs). I'm remaining logged in to VPN and monitoring in case it happens again. Unfortunately I forgot to check dmesg before a reboot, and it turns out it doesn't store syslog permanently. Way to go, D-Link. Again. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 14:17, Hamish McIntyre-Bhatty wrote: NB: On desktop I seem to have a very high number for "SUnreclaim" in /proc/meminfo: MemTotal: 32812004 kB MemFree: 8619976 kB MemAvailable: 9572924 kB Buffers: 61772 kB Cached: 1061212 kB SwapCached: 1190212 kB Active: 1939416 kB Inactive: 2347324 kB Active(anon): 1737708 kB Inactive(anon): 1529860 kB Active(file): 201708 kB Inactive(file): 817464 kB Unevictable: 796 kB Mlocked: 796 kB SwapTotal: 40943612 kB SwapFree: 38034764 kB Dirty: 412 kB Writeback: 0 kB AnonPages: 2246440 kB Mapped: 582964 kB Shmem: 105964 kB KReclaimable: 397472 kB Slab: 16832040 kB SReclaimable: 397472 kB SUnreclaim: 16434568 kB KernelStack: 28720 kB PageTables: 51072 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 57349612 kB Committed_AS: 15170832 kB VmallocTotal: 34359738367 kB VmallocUsed: 2688132 kB VmallocChunk: 0 kB Percpu: 26112 kB HardwareCorrupted: 0 kB AnonHugePages: 0 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 21204456 kB DirectMap2M: 12279808 kB DirectMap1G: 1048576 kB This isn't the same on the NAS box, but either way there are two problems to debug here and I guess they could be related. I can force the desktop to swap by using a script to allocate more than it calls "free" memory, so it's definitely not cache. The NAS box, on the other hand, seems to magic up free RAM to some extent and then eventually swaps. What does SUnreclaimable mean? Do I have a kernel memory leak? "Active" is at about 2 GB (which is roughly what I'd expect). Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 14:12, Keith Edmunds wrote: > On Thu, 10 Sep 2020 14:07:00 +0100, hamis...@live.co.uk said: > >> No difference to the output. > Surprising. Here: > > $ ps aux|wc -l > 364 > $ ps maux|wc -l > 1872 That shows a difference similar to yours, but there is no difference in those use cases eg: "ps aux | grep firefox" shows the same output as: "ps maux | grep firefox" Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 14:03, Keith Edmunds wrote: > On Thu, 10 Sep 2020 13:57:09 +0100, hamis...@live.co.uk said: > >> ps aux > Doesn't show threads. Try "ps maux" (or, if you prefer, "ps -Lef"). No difference to the output. signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 13:20, Keith Edmunds wrote: > On Thu, 10 Sep 2020 12:09:51 +0100, hamis...@live.co.uk said: > >> Using htop in that way seems to show processes that aren't currently >> running any more - like previous invocations of firefox, in the stats. >> Is that expected? > Not expected nor correct. htop shows processes that exist in the process > table. Those processes may be running, runnable, waiting on IO, sleeping > or a number of other states. Regardless of state, they will be using some > memory, although that memory may be virtual. This is strange. If I close eg Firefox, the all the (20+!) listings for that process go away. What's weird is they all have PIDs, but there aren't that many PIDs shown in the process list (or ps aux). This can't be right surely. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 13:12, Terry Coles wrote: > On Thursday, 10 September 2020 12:59:49 BST Hamish McIntyre-Bhatty wrote: >> "free" should report physical "real" memory usage right? If I'm mistaken >> there it could explain the numbers. The NAS box had only 14M or 256M >> free earlier despite showing no process using that much memory. It >> usually only uses about 50M if memory serves. > Don't forget that the memory under cache is not really in use as such; it is > simply memory containing parked data from earlier processes. The OS will > free > that if it needs it. My machine has 8G of physical memory and free -h gives: > > terry@OptiPlex:~$ free -h > totalusedfree shared buff/cache > available > Mem: 7.7Gi 6.2Gi 175Mi73Mi 1.4Gi > 1.2Gi > Swap: 15Gi 1.4Gi14Gi > > Viewing the memory in the graphical KDE System Monitor gives a totally > different answer with 6G used and no indication of cache. > > Even so, I find it interesting that you are using 21G out of 31, but > presumably > you are running lots of stuff. No, I'm running almost nothing. Over time, even with nothing new opened, it goes up and up. This is ignoring the cache. On the NAS box, I just ran a python script to allocate a load of memory, enough to force it to use swap. I made two interesting discoveries: 1) It is compressing memory to save physical RAM. 2) When I do this and then kill the process, more memory is magically reported as free for a short time. My understanding is that this shouldn't be happening, so something is being misreported? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 12:09, Hamish McIntyre-Bhatty wrote: > On 10/09/2020 12:03, Keith Edmunds wrote: >> What problem are you trying to solve? >> >> If you're just curious about memory usage, run htop, press F6 and sort by >> M_RESIDENT or, if you're feeling more adventurous, run atop and press M >> (not m). Either will show you how much memory each process is using, with >> htop giving a less detailed but possibly easier to interpret display. > I want to know why my memory usage is this high/being reported to be > this high. > > Using htop in that way seems to show processes that aren't currently > running any more - like previous invocations of firefox, in the stats. > Is that expected? > > Hamish This has just become an important issue - the same thing now seems to be happening on the model town NAS box, though I'm sure it didn't before. I reverted both our setup scripts and the river system software to old versions to no avail (including not running the river system software at all). "free" should report physical "real" memory usage right? If I'm mistaken there it could explain the numbers. The NAS box had only 14M or 256M free earlier despite showing no process using that much memory. It usually only uses about 50M if memory serves. Unfortunately the NAS box only has very basic (mostly busybox) utilities that seem to be extremely unhelpful when diagnosing this because they all report virtual memory size. Any ideas at all? I'm clutching at straw a bit here. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Very high memory usage (ignoring cache) after being powered on for days
On 10/09/2020 12:03, Keith Edmunds wrote: > What problem are you trying to solve? > > If you're just curious about memory usage, run htop, press F6 and sort by > M_RESIDENT or, if you're feeling more adventurous, run atop and press M > (not m). Either will show you how much memory each process is using, with > htop giving a less detailed but possibly easier to interpret display. I want to know why my memory usage is this high/being reported to be this high. Using htop in that way seems to show processes that aren't currently running any more - like previous invocations of firefox, in the stats. Is that expected? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Very high memory usage (ignoring cache) after being powered on for days
Hi all, Sorry for spamming the list a bit lately! :) Another issue I've had on my desktop, is that after being powered on for after maybe 4 hours, my memory usage starts to climb, but no process is listed as having used much memory. After a few days, this is what "free -h" shows: total used free shared buff/cache available Mem: 31Gi 21Gi 1.3Gi 146Mi 8.9Gi 9.6Gi Swap: 39Gi 427Mi 38Gi So total memory usage ignoring cache is roughly 31 - (8.9 + 9.6) = 12.2 GB. However, if I look in the process manager (displaying root processes too), the total is nowhere near 12.2GB, it's more like 2GB, if even that high. Any ideas how I can diagnose this? Now's a good opportunity because I'm uploading a big backup and the system is currently running having been on for days. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Very high memory usage (ignoring cache) after being powered on for days
Hi all, Sorry for spamming the list a bit lately! :) Another issue I've had on my desktop, is that after being powered on for after maybe 4 hours, my memory usage starts to climb, but no process is listed as having used much memory. After a few days, this is what "free -h" shows: total used free shared buff/cache available Mem: 31Gi 21Gi 1.3Gi 146Mi 8.9Gi 9.6Gi Swap: 39Gi 427Mi 38Gi So total memory usage ignoring cache is roughly 31 - (8.9 + 9.6) = 12.2 GB. However, if I look in the process manager (displaying root processes too), the total is nowhere near 12.2GB, it's more like 2GB, if even that high. Any ideas how I can diagnose this? Now's a good opportunity because I'm uploading a big backup and the system is currently running having been on for days. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 09/09/2020 14:42, Terry Coles wrote: > On Wednesday, 9 September 2020 14:15:08 BST Hamish McIntyre-Bhatty wrote: >> Perhaps this explains some of our issues with the network in previous >> years at the model town, Terry? It might be worth me making a note on >> the forum about it. I'm currently trying a different overlay which is >> supposed to fix the problem at the expense of a little bit of USB >> throughput. > It's certainly worth thinking about. Still having issues with my storage unmounting (this time nothing in dmesg at all), but network seems better so far. Haven't tested throughput. Any thoughts about diagnosing storage unmounting with no explanation is dmesg? This seems rather strange to me. I'm using a powered hub too now, in case I forgot to mention. Other volumes on the same external disk stay mounted. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] How to Cancel Existing systemd.timer(5) OnCalendar Values.
On 09/09/2020 12:15, Keith Edmunds wrote: > On Wed, 9 Sep 2020 09:44:44 +0100, hamis...@live.co.uk said: > >> so has anyone experienced cron jobs sporadically (kind of randomly) not >> running when they're meant to? > No, never (and I've been using Linux since the early 90s, and UNIX before > that). > > Set up your cron jobs so that the command is: > > [...] your-command 2>&1 | /usr/bin/logger -t 'my cronjob' > > Then take a look in your syslog files after the cron job has apparently > not run. > > Linux Tips: https://www.tiger-computing.co.uk/category/techtips/ > Yeah, that's why I'm asking - cron sounds bulletproof by a lot of what I've heard. Cheers, I'll do that. At the moment I'm debugging something else wrong with the pi, but I'll give this a shot once that's done. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 09/09/2020 09:41, Hamish McIntyre-Bhatty wrote: > > Cheers for finding that, I didn't really have any luck in my search for > that (or for young Skywalker). > > NB: meant "errors=remount-ro", that was a typo. > > I'm kinda surprised a networking-driver-related OOPS can cause LUKS > volumes to fail (other volumes w/ same filesystem do not get unmounted, > but this one does consistently). > > Hamish Going by what the helpful guy at https://github.com/raspberrypi/linux/issues/3843 says, this might be an issue with the underlying driver for the USB chip in the older pis. The last time it unmounted, all my USB devices reset, so that suggests this is indeed the problem, or at least part of it. Perhaps this explains some of our issues with the network in previous years at the model town, Terry? It might be worth me making a note on the forum about it. I'm currently trying a different overlay which is supposed to fix the problem at the expense of a little bit of USB throughput. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] How to Cancel Existing systemd.timer(5) OnCalendar Values.
On 09/09/2020 09:38, Ralph Corderoy wrote:
> Hi Patrick,
>
>> Is it intentional that OnCalender= has a different spelling of
>> Calend[a,e]r than OnCalendar=20:12?
> Nope! Thanks, that fixes it.
>
> $ systemctl cat updatedb.timer
> # /usr/lib/systemd/system/updatedb.timer
> [Unit]
> Description=Daily locate database update
>
> [Timer]
> OnCalendar=daily
> AccuracySec=12h
> Persistent=true
>
> # /etc/systemd/system/updatedb.timer.d/override.conf
> [Timer]
> OnCalendar=
> OnCalendar=20:12
> Persistent=no
> $
> $ systemctl show updatedb.timer | grep OnCalendar=
> TimersCalendar={ OnCalendar=*-*-* 20:12:00 ;
> next_elapse=Wed 2020-09-09 20:12:00 BST }
> $
>
> Now I just have to work out why updatedb.conf(5)'s PRUNE_BIND_MOUNTS
> doesn't seem to have an effect. :-)
>
Seems like it might be slightly related, so has anyone experienced cron
jobs sporadically (kind of randomly) not running when they're meant to?
On the same pi as has the kernel OOPS issue, I have cron jobs to turn
the power and SD access lights off at night and on at day, and run an
update script in the early morning. For a while now, these don't always
seem to run. I haven't bothered to dive into exactly why, but I was
wondering if there were any common misconfigurations that cause this.
I wouldn't find myself especially surprised if the kernel OOPS was also
somehow knocking CRON out/off-kilter.
Hamish
signature.asc
Description: OpenPGP digital signature
--
Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00
Check to whom you are replying
Meetings, mailing list, IRC, ... http://dorset.lug.org.uk
New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 09/09/2020 04:25, Ralph Corderoy wrote: > Hi Hamish, > >> [983261.923836] NETDEV WATCHDOG: enxb827eb7194f1 (lan78xx): transmit queue 0 >> timed out >> [983261.923915] WARNING: CPU: 2 PID: 0 at net/sched/sch_generic.c:466 >> dev_watchdog+0x2b0/0x2b8 > ... >> [983261.924056] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT) > ... >> Looks like a kernel oops? > Yes. > https://bugs.launchpad.net/ubuntu/+source/linux-raspi2/+bug/1861936/comments/43 > is a recent comment on a thread about it. > >> Even though the volume has the "errors=remount-rw" option, it still >> unmounts completely when this happens. Good news is no I/O errors so >> the disk is okay at least. > ‘errors=remount-rw’ is what to do when there's a media error or > something else which can be recovered. You're suffering from a device > driver failure which knocks out a lump of software. Cheers for finding that, I didn't really have any luck in my search for that (or for young Skywalker). NB: meant "errors=remount-ro", that was a typo. I'm kinda surprised a networking-driver-related OOPS can cause LUKS volumes to fail (other volumes w/ same filesystem do not get unmounted, but this one does consistently). Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 07/09/2020 14:36, Hamish McIntyre-Bhatty wrote: > On 24/06/2020 10:18, Hamish McIntyre-Bhatty wrote: >> On 20/06/2020 14:30, Patrick Wigmore wrote: >>> I wonder if it makes a difference what tools you use to unlock and >>> mount the LUKS container and partition. E.g. cryptsetup, mount, >>> udisksctl, some GUI tool (which might depend on udisks2), or something >>> else. >>> >>> The most similar thing I've recently encountered was having a USB >>> flash drive get unmounted and re-enumerated (from /dev/sda to >>> /dev/sdb) during an out-of-memory event (while running ffmpeg on a BT >>> Home Hub). >> Perhaps, I don't know. I use "cryptsetup luksOpen". >> >> Do you check dmesg(1), e.g. ‘dmesg -Hx’ then type ‘-R’, or >> journalctl(1)? >> >> It just happened again and I checked both - nothing to see about the >> device. That's why I started looking in /var/log/messages as it >> supposedly has more history, but nothing there either. >> >> Hamish > I have some new information. I've since bought a powered hub to solve > the power problem (I confirmed there was one), and it has been better > for a while until now. The other day I turned of ufw firewall logging to > keep dmesg clear so I can see what happens. I found this had happened: > > 983261.923799] [ cut here ] > [983261.923836] NETDEV WATCHDOG: enxb827eb7194f1 (lan78xx): transmit > queue 0 timed out > [983261.923915] WARNING: CPU: 2 PID: 0 at net/sched/sch_generic.c:466 > dev_watchdog+0x2b0/0x2b8 > [983261.923919] Modules linked in: xt_recent dm_crypt aes_neon_bs > aes_neon_blk crypto_simd cryptd aes_arm64 algif_skcipher af_alg 8021q > garp stp llc dm_mod brcmfmac brcmutil sg cfg80211 joydev evdev > bcm2835_codec(C) bcm2835_v4l2(C) rfkill v4l2_mem2mem > bcm2835_mmal_vchiq(C) v4l2_common videobuf2_vmalloc videobuf2_dma_contig > videobuf2_memops videobuf2_v4l2 videobuf2_common videodev > raspberrypi_hwmon media hwmon vc_sm_cma(C) uio_pdrv_genirq uio > nf_log_ipv6 ip6t_REJECT nf_reject_ipv6 xt_hl ip6_tables ip6t_rt > nf_log_ipv4 nf_log_common ipt_REJECT nf_reject_ipv4 xt_multiport xt_LOG > nft_limit xt_limit xt_addrtype xt_tcpudp xt_conntrack nft_compat > nft_counter nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp > nf_nat nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 > nf_tables nfnetlink i2c_dev > [983261.924031] ip_tables x_tables ipv6 > [983261.924051] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G > C 4.19.118-v8+ #1311 > [983261.924056] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT) > [983261.924061] pstate: 8005 (Nzcv daif -PAN -UAO) > [983261.924067] pc : dev_watchdog+0x2b0/0x2b8 > [983261.924071] lr : dev_watchdog+0x2b0/0x2b8 > [983261.924074] sp : ff8008013d70 > [983261.924077] x29: ff8008013d70 x28: ffc03c59ba00 > [983261.924083] x27: ff8008cc5000 x26: 0140 > [983261.924090] x25: x24: ffc03b24d480 > [983261.924102] x23: ffc03b24d45c x22: ffc03b306280 > [983261.924114] x21: ff8008cc6000 x20: ffc03b24d000 > [983261.924119] x19: x18: ff8008cc8688 > [983261.924125] x17: x16: > [983261.924130] x15: ff8008df8e78 x14: 74756f2064656d69 > [983261.924135] x13: 7420302065756575 x12: 712074696d736e61 > [983261.924140] x11: 7274203a29787838 x10: > [983261.924145] x9 : 00016644 x8 : > [983261.924150] x7 : ff8008cc8688 x6 : ffc03e5a20a0 > [983261.924155] x5 : x4 : fff8 > [983261.924160] x3 : x2 : 0004 > [983261.924167] x1 : f1fa254012d51300 x0 : > [983261.924179] Call trace: > [983261.924187] dev_watchdog+0x2b0/0x2b8 > [983261.924199] call_timer_fn+0x34/0x1c8 > [983261.924207] expire_timers+0xbc/0x150 > [983261.924215] run_timer_softirq+0xb4/0x1a8 > [983261.924224] __do_softirq+0x17c/0x3cc > [983261.924232] irq_exit+0xe8/0xf8 > [983261.924238] __handle_domain_irq+0x90/0x100 > [983261.924243] bcm2836_arm_irqchip_handle_irq+0x68/0xd8 > [983261.924248] el1_irq+0xb4/0x130 > [983261.924254] arch_cpu_idle+0x30/0x1f0 > [983261.924260] default_idle_call+0x24/0x40 > [983261.924265] do_idle+0x224/0x240 > [983261.924270] cpu_startup_entry+0x28/0x30 > [983261.924276] secondary_start_kernel+0x188/0x1d8 > [983261.924280] ---[ end trace b2a07353fff8d125 ]--- > > Hopefully the list won't clobber that but I'll try an attachment if it > does. Looks like a kernel oops? Even though the volume has the > "errors=remount-rw" option, it still unmounts completely when this > happens.
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 24/06/2020 10:18, Hamish McIntyre-Bhatty wrote: > On 20/06/2020 14:30, Patrick Wigmore wrote: >> I wonder if it makes a difference what tools you use to unlock and >> mount the LUKS container and partition. E.g. cryptsetup, mount, >> udisksctl, some GUI tool (which might depend on udisks2), or something >> else. >> >> The most similar thing I've recently encountered was having a USB >> flash drive get unmounted and re-enumerated (from /dev/sda to >> /dev/sdb) during an out-of-memory event (while running ffmpeg on a BT >> Home Hub). > Perhaps, I don't know. I use "cryptsetup luksOpen". > > Do you check dmesg(1), e.g. ‘dmesg -Hx’ then type ‘-R’, or > journalctl(1)? > > It just happened again and I checked both - nothing to see about the > device. That's why I started looking in /var/log/messages as it > supposedly has more history, but nothing there either. > > Hamish I have some new information. I've since bought a powered hub to solve the power problem (I confirmed there was one), and it has been better for a while until now. The other day I turned of ufw firewall logging to keep dmesg clear so I can see what happens. I found this had happened: 983261.923799] [ cut here ] [983261.923836] NETDEV WATCHDOG: enxb827eb7194f1 (lan78xx): transmit queue 0 timed out [983261.923915] WARNING: CPU: 2 PID: 0 at net/sched/sch_generic.c:466 dev_watchdog+0x2b0/0x2b8 [983261.923919] Modules linked in: xt_recent dm_crypt aes_neon_bs aes_neon_blk crypto_simd cryptd aes_arm64 algif_skcipher af_alg 8021q garp stp llc dm_mod brcmfmac brcmutil sg cfg80211 joydev evdev bcm2835_codec(C) bcm2835_v4l2(C) rfkill v4l2_mem2mem bcm2835_mmal_vchiq(C) v4l2_common videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops videobuf2_v4l2 videobuf2_common videodev raspberrypi_hwmon media hwmon vc_sm_cma(C) uio_pdrv_genirq uio nf_log_ipv6 ip6t_REJECT nf_reject_ipv6 xt_hl ip6_tables ip6t_rt nf_log_ipv4 nf_log_common ipt_REJECT nf_reject_ipv4 xt_multiport xt_LOG nft_limit xt_limit xt_addrtype xt_tcpudp xt_conntrack nft_compat nft_counter nf_conntrack_netbios_ns nf_conntrack_broadcast nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables nfnetlink i2c_dev [983261.924031] ip_tables x_tables ipv6 [983261.924051] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G C 4.19.118-v8+ #1311 [983261.924056] Hardware name: Raspberry Pi 3 Model B Plus Rev 1.3 (DT) [983261.924061] pstate: 8005 (Nzcv daif -PAN -UAO) [983261.924067] pc : dev_watchdog+0x2b0/0x2b8 [983261.924071] lr : dev_watchdog+0x2b0/0x2b8 [983261.924074] sp : ff8008013d70 [983261.924077] x29: ff8008013d70 x28: ffc03c59ba00 [983261.924083] x27: ff8008cc5000 x26: 0140 [983261.924090] x25: x24: ffc03b24d480 [983261.924102] x23: ffc03b24d45c x22: ffc03b306280 [983261.924114] x21: ff8008cc6000 x20: ffc03b24d000 [983261.924119] x19: x18: ff8008cc8688 [983261.924125] x17: x16: [983261.924130] x15: ff8008df8e78 x14: 74756f2064656d69 [983261.924135] x13: 7420302065756575 x12: 712074696d736e61 [983261.924140] x11: 7274203a29787838 x10: [983261.924145] x9 : 00016644 x8 : [983261.924150] x7 : ff8008cc8688 x6 : ffc03e5a20a0 [983261.924155] x5 : x4 : fff8 [983261.924160] x3 : x2 : 0004 [983261.924167] x1 : f1fa254012d51300 x0 : [983261.924179] Call trace: [983261.924187] dev_watchdog+0x2b0/0x2b8 [983261.924199] call_timer_fn+0x34/0x1c8 [983261.924207] expire_timers+0xbc/0x150 [983261.924215] run_timer_softirq+0xb4/0x1a8 [983261.924224] __do_softirq+0x17c/0x3cc [983261.924232] irq_exit+0xe8/0xf8 [983261.924238] __handle_domain_irq+0x90/0x100 [983261.924243] bcm2836_arm_irqchip_handle_irq+0x68/0xd8 [983261.924248] el1_irq+0xb4/0x130 [983261.924254] arch_cpu_idle+0x30/0x1f0 [983261.924260] default_idle_call+0x24/0x40 [983261.924265] do_idle+0x224/0x240 [983261.924270] cpu_startup_entry+0x28/0x30 [983261.924276] secondary_start_kernel+0x188/0x1d8 [983261.924280] ---[ end trace b2a07353fff8d125 ]--- Hopefully the list won't clobber that but I'll try an attachment if it does. Looks like a kernel oops? Even though the volume has the "errors=remount-rw" option, it still unmounts completely when this happens. Good news is no I/O errors so the disk is okay at least. Any ideas on how I might further debug this? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-10-06 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Loss of OS during upgrade
On 02/09/2020 17:19, C Wills wrote: > Interesting meeting last night especially about the keyboard settings, > found several things about the PC keyboard I was using and it's > additional keys. > > Sorry this is a long email. > Now to my problem: While trying to upgrade from Mint 19.3 to Mint 20 > problems arose which have stop me using the Laptop (I'm on our PC for > now). > Long story so cutting it short. During the upgrade I used the 'other' > option as the laptop has 4 partitions:- sda1= OS (Mint) system files, > sda2=swop, sda3=Home, sda4=odds area. > Previous upgrades I've reformatted sda1 and left all other 'as is' and > it's worked every time (till now). > Now sda1 is empty (no files to be seen), all others still OK. > Fault now says ' no UEFI file to be found' (or similar words) > With Ralphs help we have established that the BIOS does support UEFI > and it uses MBR(?) for Grub2. > > Can now only use a Live Disk to operate the Laptop. Gparted shows > partitions and confirms OS is empty. > > From a 'live disk' would like to copy sda3 to another disk (USB 160Gb > with 2 partitions - sdb1=NTFS and sdb2= Ext4 140Gb) before trying to > re-install and ignoring the warnings!. > BUT > I can only copy sda3 to any /other/ location (sdb2) by becoming 'Root' > and when I've done this before I've ended up with all Directories and > files being 'Owned' by Root and lost 'my' ownership. > > How can I copy sda3 to sdb2 using a 'live disk' conserving all > properties.. All programs I've got don't seems to allow this. I'm > normally a GUI user not terminal user unless I have a command to copy > exactly, any help please? > > PS Laptop works fine from the live DVD ISO's of both Mint versions. > Hello Clive, Do you have a backup you can restore before attempting to upgrade again? This would probably be the easiest option. I would also recommend using the "mintupgrade" tool, as suggested by the Mint team at https://linuxmint-user-guide.readthedocs.io/en/latest/upgrade-to-mint-20.html A long shot, but did you happen to have the 32-bit version of Linux Mint 19.3 installed? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Understanding EXT4 disk options
On 31/08/2020 12:33, Ralph Corderoy wrote: > Do a remount changing an option line barrier between 0 and 1 and back > again. Each time observe whether mount and mountinfo reflect the latest > change and whether they tally. This should give you confidence in what > to trust when observing your final desired settings. > Yeah, it does seem to do that. Thanks for your help. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Next Online Meeting - Tonight at 8pm
I have a shift this evening, so sadly I can't attend unless you're still going at 10PM when I get back. Hamish On 01/09/2020 10:17, Terry Coles wrote: > All, > > The next Online Meeting is tonight at 8 pm using Jitsi. > > For those who missed it earlier; here are the Instructions for joining: > > Simply click on the following link and you will be taken to the Meeting > using your default browser: > > https://meet.jit.si/dorset-lug > > Chrome or Chromium are probably better than Firefox for using Jitsi. An > alternative to installing one of those two is to obtain it bundled especially > for Jitsi from: > > https://github.com/jitsi/jitsi-meet-electron/releases > > This should be as simple as > > wget -q https://github.com/jitsi/jitsi-meet-electron/releases/download/ > v2.3.1/jitsi-meet-x86_64.AppImage > chmod +x jitsi-meet-x86_64.AppImage > ./jitsi-meet-x86_64.AppImage > > and then entering ‘dorset-lug’ as the meeting ID. > > Hope to see you all this evening. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Understanding EXT4 disk options
I believe they have: if I just run "mount", it shows the new options that I specified. I know they work because I did encounter the "mount point not mounted or bad option." error when I was trying to figure out what options it would accept. I reckon that because this is D-Link's stupid flavour of Linux (with a 2.6 kernel, not even the latest 2.6 kernel either) that it just doesn't behave as you'd expect it to. I have found it to behave oddly in many different respects. I have observed faster disk write speeds after changing the mount options (significant - 20% quicker for sustained contiguous writes), so I'm reasonably sure they have applied, but I could be wrong. Is there another different way to check? Hamish On 29/08/2020 09:42, Ralph Corderoy wrote: > Hello Hamish, > >> Good idea, but sadly that just shows the default mount options. > Then that suggests they haven't been changed, at least if that old > kernel matches mine here. > > Make an ext4 filesystem and mount point. > > $ dd if=/dev/zero bs=2M count=1 of=fs.img > 1+0 records in > 1+0 records out > 2097152 bytes (2.1 MB, 2.0 MiB) copied, 0.0164876 s, 127 MB/s > $ mkfs -t ext4 fs.img > mke2fs 1.45.2 (27-May-2019) > Discarding device blocks: done > Creating filesystem with 2048 1k blocks and 256 inodes > > Allocating group tables: done > Writing inode tables: done > Creating journal (1024 blocks): done > Writing superblocks and filesystem accounting information: done > > $ mkdir mnt > $ > > Mount it with duff options. > > $ sudo mount -o loop,data=writeback,barrier=0,commit=30 fs.img mnt > $ fgrep $PWD /proc/$$/mountinfo > 753 193 7:1 / /home/tmp/1598690175.392785701/mnt rw,relatime shared:408 - > ext4 /dev/loop1 rw,nobarrier,commit=30,data=writeback > $ > > Attempt to change all of them to sane ones. > > $ sudo mount -o loop,remount,data=ordered,barrier=1,commit=5 fs.img mnt > mount: /home/tmp/1598690175.392785701/mnt: mount point not mounted or bad > option. > $ fgrep $PWD /proc/$$/mountinfo > 753 193 7:1 / /home/tmp/1598690175.392785701/mnt rw,relatime shared:408 - > ext4 /dev/loop1 rw,nobarrier,commit=30,data=writeback > $ > > None of them change. It's because it's not possible to change ‘data’ > with remount. > > Change the other two options. > > $ sudo mount -o loop,remount,barrier=1,commit=5 fs.img mnt > $ fgrep $PWD /proc/$$/mountinfo > 753 193 7:1 / /home/tmp/1598690175.392785701/mnt rw,relatime shared:408 - > ext4 /dev/loop1 rw,data=writeback > $ > > Try again to change the remaining one. > > $ sudo mount -o loop,remount,data=ordered fs.img mnt > mount: /home/tmp/1598690175.392785701/mnt: mount point not mounted or bad > option. > $ fgrep $PWD /proc/$$/mountinfo > 753 193 7:1 / /home/tmp/1598690175.392785701/mnt rw,relatime shared:408 - > ext4 /dev/loop1 rw,data=writeback > $ > > Still doesn't work. > > $ sudo umount mnt > $ > > So if... > >> Good idea, but sadly that just shows the default mount options. > ...then it suggests none of your remount-option changes have taken > effect? > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Understanding EXT4 disk options
Good idea, but sadly that just shows the default mount options. It does show some other very small (unused) HDD partitions are mounted with data=ordered though. On 28/08/2020 20:39, Ralph Corderoy wrote: > Hello Hamish, > >> I was able to specify all options except the "data=ordered" one, as it >> rejects all values except writeback. However, the default seems to be >> ordered, so it should be okay. > ‘cat /proc/$$/mountinfo’ may show the options in use to help confirm > ordered is present, or that writeback isn't there. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Understanding EXT4 disk options
Managed to get it working. I was able to specify all options except the "data=ordered" one, as it rejects all values except writeback. However, the default seems to be ordered, so it should be okay. Ironically, disk performance seems slightly better now as well. Hamish On 27/08/2020 14:22, Hamish McIntyre-Bhatty wrote: > Okay, I'll try disabling that option too. I need to devise a way of > checking that the storage is still working properly after changing the > options - I don't trust D-Link (the manufacturer) very much at this stage XD > > Cheers Ralph, > > Hamish > > On 27/08/2020 14:19, Ralph Corderoy wrote: >> Hello Hamish, >> >>> The other option that gets my attention is "stripe=96", but I'm not >>> sure I understand what that does in a RAID 1 array. Is this another >>> option I should change? >> ext4(5) says >> >> stripe=n >> Number of filesystem blocks that mballoc will try to use for >> allocation size and alignment. For RAID5/6 systems this should >> be the number of data disks * RAID chunk size in filesystem >> blocks. >> >> RAID 5 and 6 are https://en.wikipedia.org/wiki/Raid_5#RAID_5 >> >> Your NAS is RAID 1 which is a simple mirror so striding doesn't really >> make sense and you should leave mballoc() to its default behaviour. >> >> AIUI, striding tries to even wear and increase performance by evenly >> spreading ‘hot’, that is frequently read/written, filesystem metadata >> across multiple disks where each disk is storing a fragment of the >> information. >> >> signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Understanding EXT4 disk options
Okay, I'll try disabling that option too. I need to devise a way of checking that the storage is still working properly after changing the options - I don't trust D-Link (the manufacturer) very much at this stage XD Cheers Ralph, Hamish On 27/08/2020 14:19, Ralph Corderoy wrote: > Hello Hamish, > >> The other option that gets my attention is "stripe=96", but I'm not >> sure I understand what that does in a RAID 1 array. Is this another >> option I should change? > ext4(5) says > > stripe=n > Number of filesystem blocks that mballoc will try to use for > allocation size and alignment. For RAID5/6 systems this should > be the number of data disks * RAID chunk size in filesystem > blocks. > > RAID 5 and 6 are https://en.wikipedia.org/wiki/Raid_5#RAID_5 > > Your NAS is RAID 1 which is a simple mirror so striding doesn't really > make sense and you should leave mballoc() to its default behaviour. > > AIUI, striding tries to even wear and increase performance by evenly > spreading ‘hot’, that is frequently read/written, filesystem metadata > across multiple disks where each disk is storing a fragment of the > information. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Understanding EXT4 disk options
I like the solution of remounting. I'm running an extended SMART test on the drives, but after that I'll give it a shot. The other option that gets my attention is "stripe=96", but I'm not sure I understand what that does in a RAID 1 array. Is this another option I should change? Hamish On 27/08/2020 13:15, Ralph Corderoy wrote: > Hello Hamish, > >> * The commit interval is set to 30 seconds instead of the default 5. >> * The data mode is set to writeback >> * barrier is set to 0, which, from what I understand, can also >> compromise data consistency. > The NAS is going for maximum performance and never mind the integrity of > the data. Could you remind us of the manufacturer. :-) > >> My understanding of these issues from what I read here >> https://www.kernel.org/doc/html/latest/admin-guide/ext4.html#options >> is that this effectively disables journaling, and seriously impacts >> the reliability of the storage system (RAID 1). > The journal will still be used, but in a way which makes recovery less > robust. > > ext4(5) also describes the options and the defaults of > ‘data=ordered,barrier=1,commit=5’ are much better. Certainly normal for > a database's filesystem, or anything where data loss is to be minimised. > > And RAID 1 doesn't help because these options are all about the > higher-level meaning of the data sent to the mirroring drives so > assuming the RAID 1 does it's job then you've multiple reliable copies > of incorrect data. :-) > >> Unfortunately, it is D-Link's scripts that mount the RAID array, so >> I doubt that I can change the options. > Look at ‘remount’ in mount(8). Not as part of binding, but standalone. > With the right arguments you should be able to change the mount options > to more sane ones soon after boot, with it ignoring /etc/fstab's > settings. > signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Understanding EXT4 disk options
Hello, After a power loss at the model town recently, I discovered the NAS box's database had become corrupted. Fortunately, it was able to recover the lost data with the "REPAIR TABLE" SQL command. Unfortunately, when I was investigating the cause, I found that the following mount options are being used for the storage: * The commit interval is set to 30 seconds instead of the default 5. * The data mode is set to writeback * barrier is set to 0, which, from what I understand, can also compromise data consistency. My understanding of these issues from what I read here https://www.kernel.org/doc/html/latest/admin-guide/ext4.html#options is that this effectively disables journaling, and seriously impacts the reliability of the storage system (RAID 1). Unfortunately, it is D-Link's scripts that mount the RAID array, so I doubt that I can change the options. After a quick call with Terry, I had the idea of running the "sync" command regularly as a kind of kludgy workaround, in addition to automatically repairing the database tables on each start. I'm not particularly happy with either of these solutions. Does anyone else have any ideas? I know there are some smart people here so I figure it's worth asking. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Computing with photons instead of electrons
Hi, Thought some of you might find this interesting too :) https://www.tomshardware.com/news/lightmatter-mars-soc-bends-light-to-process-data-silicon-photonics Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Waveshare 1.5 inch OLED Display
On 05/08/2020 10:20, Terry Coles wrote: > That's what I thought. > I intend to share this with the volunteer who originally bought the display, > He was / is a hardware engineer before he retired, so we might be able to get > quite a bit out of this between us. > > Hamish has been busy on the River System project so hasn't really been > involved in this, but he's welcome to join in if he has time :-) > I'd be happy to help, but as you say I'm quite busy at the moment. Once I've got a few things out of the way I should have more time, but it looks like you've got it covered. I'm sure you told me, but what's the screen going to be used for? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-09-01 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] World Community Grid COVID-19 project
I'm fairly sure this has already been mentioned, but if not: World Community Grid has launched a COVID-19 projects as well, and it runs on ARM-based CPUs and Android phones as well as x86. Ibercivis has stopped as far as I can tell, so this might be a good replacement project :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] BOINC and Folding on Ubuntu 20.04 and derivatives
On 23/07/2020 17:41, Terry Coles wrote: > On Thursday, 23 July 2020 17:29:45 BST Hamish McIntyre-Bhatty wrote: >> I was wondering if any of you have run BOINC and/or Folding on Ubuntu >> 20.04 or a derivative. I'm thinking of Upgrading my desktop from Mint >> 19.3 to 20 soon, and it'd be good to know if I'm likely to experience >> problems with these first :) > I'm running BOINC on Kubuntu 20.04. The client runs well, but I had problems > with BOINC Manager because it occasionally refused to load after a reboot. I > fixed it by adding a '-a' switch to the launcher in Plasma: > > boincmgr -a > > According to the man page, this tells the BOINC Manager that it was started > by > the operating system automatically. > > On odd occasions, I have found that I had to start the Manager with sudo, to > start another instance. Closing it again seems to allow me to launch in the > normal way. > > As a precaution, I usually close the BOINC Manager before shutting down. > In case anyone else is interested, both BOINC and Folding work for me, but the Folding client GUI doesn't because it uses old Python 2 stuff. The solution is to use the web client instead, which works just fine. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] BOINC and Folding on Ubuntu 20.04 and derivatives
Hi all, I was wondering if any of you have run BOINC and/or Folding on Ubuntu 20.04 or a derivative. I'm thinking of Upgrading my desktop from Mint 19.3 to 20 soon, and it'd be good to know if I'm likely to experience problems with these first :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Determining which revision of code was downloaded from GitLab.
On 20/07/2020 14:04, Terry Coles wrote: > On Monday, 20 July 2020 13:55:14 BST Hamish McIntyre-Bhatty wrote: >> No, I think we absolutely could do that, and it would be a good >> solution, though perhaps a little tricky for future maintainers. If you >> think we should do that then I'm happy to write us such a script. > I think that it should be OK for future maintainers, providing that we > document the purpose of the script in some prominent place. Okay :) I guess I'll write it in Python to make it at least slightly more cross-platform in case future volunteers aren't running Linux. >> However, I was looking to a solution to the existing problem of not >> knowing exactly what revision/commit we have deployed :) > I think that horse has probably already bolted. If lockdown hadn't occurred, > we wouldn't have ended up forgetting where we were (that is no way intended > to > be an excuse for what happened). Still it has happened and we have > established, by looking at the code, that one particular release had made it > into the deployed code. We should be able to get pretty close by a > combination of examining the diffs in Gitlab and looking at the commentary on > the Forum. > > I don't think we ever deployed any software without some kind of discussion > beforehand. > These things happen, it's no big deal :) As long as we can get reasonably close it shall be fine, and as you say this shouldn't be too tricky. signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Determining which revision of code was downloaded from GitLab.
On 20/07/2020 13:45, Terry Coles wrote: > On Monday, 20 July 2020 13:35:19 BST Hamish McIntyre-Bhatty wrote: >> Sadly, seeing as this is Python software, we don't have a makefile. I >> know there are a variety of ways to solve this going forwards, including >> even just noting what the commit was at the time of deployment on our >> forum or something like that, but this is more about how to solve the >> present issue. Sadly, I think the tarball might just not include any >> revision details. >> >> Whatever we do, it needs to be as simple as possible to ease the life of >> future maintainers - things like Makefiles and symlinks are probably not >> ideal, but thanks for the ideas :) > Presumably, when the makefile described by Ralph processes this, it creates > this zip through some kind of script. > > Couldn't we just create a 'Release' script that works on a text file > containing > the full list of files in the system (as a makefile does) fetches them from > GitLab, zips them up and names the resulting archive with the Revision Number? > > In other words a makefile, but without the compilation stage. > > Have I missed something? > No, I think we absolutely could do that, and it would be a good solution, though perhaps a little tricky for future maintainers. If you think we should do that then I'm happy to write us such a script. However, I was looking to a solution to the existing problem of not knowing exactly what revision/commit we have deployed :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Determining which revision of code was downloaded from GitLab.
On 20/07/2020 12:30, Ralph Corderoy wrote: > > If the tar-file download doesn't include the revision in its filename > and the directory it unpacks then I'd either post-process it to do that > or stop using the web download and do a proper ‘release’ target in the > software's makefile. Then when you want a release to ship you > > make REV=12345678ab release > > and it creates a ‘wmt-12345678ab.tar.lzip’ or similar which is a > check-out of that revision from the public repo into directory > ‘wmt-12345678ab’, all tar'd up. > > You can ship those tar files about, unpack them, and then alter a ‘wmt’ > symbolic link to move between which version will be used, allowing for > easy downgrade on a machine with no Internet. > > Code can inspect the ‘real’ path to its location to determine its > version for logging, etc. > Sadly, seeing as this is Python software, we don't have a makefile. I know there are a variety of ways to solve this going forwards, including even just noting what the commit was at the time of deployment on our forum or something like that, but this is more about how to solve the present issue. Sadly, I think the tarball might just not include any revision details. Whatever we do, it needs to be as simple as possible to ease the life of future maintainers - things like Makefiles and symlinks are probably not ideal, but thanks for the ideas :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Determining which revision of code was downloaded from GitLab.
Hi all, So as you may know, GitHub and GitLab allow you to download all files from a branch as a compressed archive, using your web browser. For a while, this is how we've deployed the river control system software at Wimborne Model Town, as the system hasn't had internet access, historically speaking. As we're now almost ready for some new deployments, it'd be useful to know exactly what revision of code the pis contain, so we can slowly bring things up to date. I was wondering if anyone know whether GitLab's file downloads contain any kind of git signature/watermark to show you what commit you downloaded, or if there's a way to enable this. According to Terry, Microsoft SourceSafe (?) did this, so I'm sure there's a way of achieving this with git too, but I have been unable to find it. Any ideas? :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Making iptables Entries Persistent
On 19/07/2020 20:10, Patrick Wigmore wrote: > On Sat, 18 Jul 2020 18:17:38 +0100, Terry Coles wrote: >> Hi, >> >> It has been suggested that I add an iptables rule into some devices >> and make it persistent by adding the rule to /etc/rc.local. >> >> I naively thought that iptables rules were persistent, but a quick >> google throws up the idea of using iptables-save/iptables-restore >> but also iptables- persistent. >> >> Is there a right way? > I wanted to know the answer to this a while ago, and I concluded that > it doesn't matter enormously. As far as I could tell, it's a bring- > your-own-persistence party and there is no one best way of doing it. > > It seems as though iptables-based firewall utilities are as numerous > as text editors and desktop environments. > > Fundamentally, you've just got to make sure that, at some sensible > moment during start-up, some commands; none in particular; will get > run that will create the rule for you. iptables-restore is one way to > do that, which might be helpful, so is iptables-persistent. Or, you > could just as well run the commands that you originally used to create > the rule. > > My solution was to write an init script that created my iptables > rules, with the rules I wanted hard-coded into the script in a manner > that was easily-editable. I thought that was a relatively neat way of > doing it, but it's certainly not the only way. I might not have done > it that way if I only wanted to load one simple rule. > > (For systemd, I suppose you would write a systemd unit instead.) > > If you were going to invest a lot of time in writing rules or scripts, > nftables might be more futureproof than iptables. But for quick, > simple rules, I wouldn't worry about that too much. > > Patrick > Sounds interesting :) For my laptop and desktop I use Gufw, a GUI frontend for Ufw "Uncomplicated FireWall", which is itself based on iptables IIRC. But I guess the Pis don't run GUIs, so using Gufw would probably not be ideal (also the interface is rather unintuitive). Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Making iptables Entries Persistent
On 18/07/2020 18:17, Terry Coles wrote: > Hi, > > It has been suggested that I add an iptables rule into some devices and make > it persistent by adding the rule to /etc/rc.local. > > I naively thought that iptables rules were persistent, but a quick google > throws up the idea of using iptables-save/iptables-restore but also iptables- > persistent. > > Is there a right way? Hi Terry, I've personally used the iptables-save/iptable-restore method and see it used more often, but I have no idea whether it's been/being replaced with iptables-persistent. Perhaps either is suitable? Someone who knows more than me can probably provide a better answer. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-08-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
On 20/06/2020 14:30, Patrick Wigmore wrote: > I wonder if it makes a difference what tools you use to unlock and > mount the LUKS container and partition. E.g. cryptsetup, mount, > udisksctl, some GUI tool (which might depend on udisks2), or something > else. > > The most similar thing I've recently encountered was having a USB > flash drive get unmounted and re-enumerated (from /dev/sda to > /dev/sdb) during an out-of-memory event (while running ffmpeg on a BT > Home Hub). Perhaps, I don't know. I use "cryptsetup luksOpen". Do you check dmesg(1), e.g. ‘dmesg -Hx’ then type ‘-R’, or journalctl(1)? It just happened again and I checked both - nothing to see about the device. That's why I started looking in /var/log/messages as it supposedly has more history, but nothing there either. Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Internet Access Using Tethering
The only thing I know related to this is that Android devices sometimes have a bug where you have to run a shell command on them to get tethering working (I did on mine), but clearly that's not the problem you're having seeing as general internet access is working. Ignoring the ping, can you connect to the VPN server through the router? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Filesystem in a LUKS volume unmounts randomly on Raspberry Pi
Hi there, I've been wondering if any of you have encountered something like this. I have a Raspberry Pi 3 B+ set up as a NAS server with an external HDD and a beefy power supply. There are multiple partitions of this drive: - /dev/sda1 is a 720GB EXT4 partition for general storage. - /dev/sda2 is a 11.2GB swap partition (the Pi doesn't have much RAM and I'd rather not wear out the SD card using it as swap). - /dev/sda3 is a 200GB LUKS container (/dev/dm-0) with an EXT4 partition inside, which I use for syncing files between my laptop and desktop with Nextcloud. This is generally very quick and reliable, but the EXT4 filesystem inside the LUKS container randomly (can be an hour, can be 3 weeks) unmounts, and there don't seem to be any errors in /var/log/messages pertaining to this. The other filesystems always stay mounted though. I haven't bothered to investigate this despite it going on for ages because I haven't yet encountered any data corruption and periodically remounting the filesystem isn't much of a hassle. My first thought is power (cos I'm not using a powered USB hub), but I would have thought that'd knock the other partitions out as well, at least some of the time. Other things could be that in using the 64-bit kernel so I can do distributed computing on the Pi, but this has been going on since way before I started doing that. Perhaps there's a bug in whatever drives LUKS storage on the Pi/its architecture? Any ideas? Hamish McIntyre-Bhatty signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Ibercivis Down?
On 18/06/2020 10:06, Terry Coles wrote: > On Thursday, 18 June 2020 10:01:13 BST Hamish McIntyre-Bhatty wrote: >> Their BOINC project page seems like it was never fully set up anyway - >> eg password reset doesn't work - so it's probably just temporary >> downtime. Anyone know if we can report it somehow? > Are you able to get there? I just get 'Loading' with the circulate icon then > it times out. No, it's just hanging for me too. signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Ibercivis Down?
On 18/06/2020 06:10, Terry Coles wrote: > I've been unable to upload any Tasks for a day or two and the Ibercivis > website appears to be down. > > Has anyone else seen this? I can't find any references to the project being > off-line on the web. > > I have a dozen completed tasks and I don't know whether to cut my losses and > focus on Rosetta. > I have the same problem - I have things waiting in a queue to upload that keep being delayed. Might as well just leave them in the queue I think - you can't get new tasks if the project is down so no change of doing more work you can't submit until they've fixed it. Their BOINC project page seems like it was never fully set up anyway - eg password reset doesn't work - so it's probably just temporary downtime. Anyone know if we can report it somehow? Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Selling GPL software with different price points.
On 10/06/2020 17:33, Patrick Wigmore wrote: > On Wed, 10 Jun 2020 15:56:41 +0100, Ralph Corderoy wrote: >> Hamish could distinguish the sold editions with ‘Personal Edition’ >> and ‘Business Edition’ though, e.g. title bars, or About dialogue >> windows? This wouldn't be a restriction, just an indication of >> which binary was bought? > That seems like a fairly reasonable interpretation. I was mostly > perturbed by the exact wording "Personal Use". > > "Personal Edition" and "Business Edition" seem OK to me, along with > things like "Golf Edition", "Canine Edition" and "Astrophysics > Edition". (I have no idea what kind of software this is.) Yeah, I agree calling them "Edition" seems better to me. For reference, this is disk erase verification software (more information at https://www.hamishmb.com/html/diskverifier.php if you want it) :) Hamish signature.asc Description: OpenPGP digital signature -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
