Hi, The skin has not been modified in any way. In my long description of the problem ;) I forgot to write that after automatic upgrade from 4.10.2 to 5.02 and this method did not work, I uninstalled the entire weewx (sudo apt purge) and installed it fresh. Later, I just copied the entries from the old configuration file to the new one. And not replacing this file, just individual entries. No files were edited or changed except the configuration file. I realize that changing from weewx to root shouldn't increase CPU usage. I only marked it to describe the problem with port 80, on which the raspberry must listen for data from the weather station.
Br Bartosz sobota, 1 czerwca 2024 o 18:21:13 UTC+2 vince napisał(a): > oops - thank you. I missed that link in the long wording of the problem > description. Like the new weectl debug output too :-) > > What I see is he's using the old original wview-compatible schema, so that > certainly leans toward the db schema issue. Again, I'd suggest temporarily > switching to the old hard-coded Standard or Smartphone or Mobile skin (as a > test) to see if the problem goes away. > > Is there a way to instrument a test version of something under the hood in > order to log which (if any) xtypes are being calculated due to not being > found in the db ? This issue seems to be hitting quite a few legacy > users. Would some test instrumentation perhaps help folks identify what's > missing in their setup ? > > On Saturday, June 1, 2024 at 9:02:14 AM UTC-7 Tom Keffer wrote: > >> Vince, he posted the output of weectl debug, which includes the schema >> and weewx.conf. >> >> -tk >> >> >> On Sat, Jun 1, 2024 at 8:26 AM vince <vince...@gmail.com> wrote: >> >>> If you have 6 years of data you probably are using the old >>> wview-compatible schema, so this 'sounds' like you are missing db elements >>> referenced in your skins. You are also running on a pi 3b (slow) and >>> mentioned you have a 2 minute archive period (a little fast) making the pi >>> work harder than usual. >>> >>> As a quick test, switch to the simplest available skin (Mobile or >>> Smartphone or Standard) and see if the problem goes away. If so it is >>> almost certainly missing elements in your database. >>> >>> Can you post your Seasons skin.conf, weewx.conf (with no >>> usernames/passwords in it), and your db schema ? >>> >>> On Saturday, June 1, 2024 at 7:52:50 AM UTC-7 Tom Keffer wrote: >>> >>>> Permission problems tend to be "all or nothing", so I doubt running as >>>> 'root' (as opposed to 'weewx') has anything to do with the excessive use >>>> of >>>> the CPU. >>>> >>>> The problem is almost surely caused by the need to calculate an >>>> aggregate of an xtype that is not in the database (as described in this >>>> wiki article <https://github.com/weewx/weewx/wiki/XTypes-performance>). >>>> Because this involves many small queries to the database, using a remote >>>> database can really exacerbate the problem. >>>> >>>> I see that you are using the Seasons skin. Have you modified it at all? >>>> To check, you may want to use the stock Seasons skin and compare times. >>>> >>>> >>>> >>>> On Sat, Jun 1, 2024 at 6:25 AM Bartosz Francman <bfra...@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> For several months I have been postponing the upgrade from version >>>>> 4.10.2-1 to version 5.02. To tell you the truth, I didn't even know that >>>>> it >>>>> was a jump from 4.x to 5.x and a few days ago I did an automatic upgrade >>>>> (sudo apt update & upgrade). >>>>> When asked about saving the configuration file, I left the old >>>>> version, as everything worked so far. >>>>> My system consists of a RaspberryPi 3B, a Eurochron EFWS 2900 station >>>>> and a database (MariaDB 10) located on the NAS so as not to write data to >>>>> the memory card. This system has been operating for 6 years (the first >>>>> entries in the database are from 2018). So far I haven't had any >>>>> problems. >>>>> Everything worked as it should. >>>>> As the weather station only works via WiFi, I had to use the >>>>> Interceptor controller to read data from the station. And unfortunately, >>>>> only port 80 could be used. It is not possible to configure the station, >>>>> apart from turning it on and setting access data to 3 weather services. >>>>> Now I will briefly describe the problems with the upgrade. >>>>> Port 80 cannot be used by any user other than root, so I had to change >>>>> the way weewx runs from the weewx user to the root user. Data began to >>>>> appear and the script did not end with the lack of access to port 80. I >>>>> thought that it was not so bad. >>>>> However, I saw that the processes related to weewx were using 100% of >>>>> the load of two raspberry processors. I found information on this group >>>>> that it may be a problem with the lack of appropriate columns in the >>>>> archive table ( >>>>> https://groups.google.com/g/weewx-user/c/6rl2FIbqVp4/m/S0Ek9ZVaBwAJ) >>>>> I did everything according to this post, plus from >>>>> https://weewx.com/docs/5.0/utilities/weectl-database/#update-a-database >>>>> update the database and check it. All this took about 2 hours due to the >>>>> large amount of data I have in this database. By the way, apart from the >>>>> high CPU load, I noticed that every half a minute there is a very large >>>>> data read from the database. And it's related to weewx. I thought that >>>>> rebuilding the database would reduce the CPU load and eliminate the >>>>> frequent polling, but unfortunately it didn't help. I don't have any very >>>>> advanced visualizations, just a basic diagram. >>>>> I left it because there is nothing else to do with this raspberry and >>>>> two out of four processors can be fully loaded. But unfortunately, about >>>>> two hours after starting the system, I received information from one >>>>> weather service that I was not sending data to them. I tried to access >>>>> this >>>>> raspberry via ssh, but it was impossible. Connection timed out. Although >>>>> the raspberry was running. >>>>> Only today I sat down to work on it and managed to connect to the >>>>> raspberry. The last log entries after issuing the sudo systemctl status >>>>> weewx command are provided below: >>>>> >>>>> pi@raspberry-pi:~ $ sudo systemctl status weewx >>>>> × weewx.service - WeeWX >>>>> Loaded: loaded (/lib/systemd/system/weewx.service; enabled; >>>>> preset: enabled) >>>>> Active: failed (Result: signal) since Sat 2024-06-01 10:36:12 >>>>> CEST; 1min 2s ago >>>>> Duration: 19h 10min 2.367s >>>>> Docs: https://weewx.com/docs >>>>> Process: 720 ExecStart=weewxd /etc/weewx/weewx.conf (code=killed, >>>>> signal=KILL) >>>>> Main PID: 720 (code=killed, signal=KILL) >>>>> CPU: 1h 59min 21.575s >>>>> >>>>> maj 31 17:51:09 raspberry-pi weewxd[720]: INFO weewx.restx: >>>>> PWSWeather: Published record 2024-05-31 17:48:00 CEST (1717170480) >>>>> maj 31 17:51:22 raspberry-pi weewxd[720]: INFO weewx.restx: >>>>> Wunderground-RF: Published record 2024-05-31 17:48:47 CEST (1717170527) >>>>> maj 31 17:51:28 raspberry-pi weewxd[720]: ERROR weewx.restx: WOW: >>>>> Failed to publish record 2024-05-31 17:48:00 CEST (1717170480): Failed >>>>> upload after 3 tries >>>>> maj 31 17:51:28 raspberry-pi weewxd[720]: INFO weewx.restx: >>>>> Wunderground-RF: Published record 2024-05-31 17:49:19 CEST (1717170559) >>>>> maj 31 17:51:53 raspberry-pi weewxd[720]: INFO weewx.restx: >>>>> Wunderground-RF: Published record 2024-05-31 17:49:51 CEST (1717170591) >>>>> maj 31 17:52:13 raspberry-pi weewxd[720]: INFO weewx.restx: >>>>> Wunderground-RF: Published record 2024-05-31 17:50:23 CEST (1717170623) >>>>> maj 31 17:52:56 raspberry-pi weewxd[720]: INFO weewx.manager: Added >>>>> record 2024-05-31 17:50:00 CEST (1717170600) to database 'weewx_metric' >>>>> cze 01 10:36:12 raspberry-pi systemd[1]: weewx.service: Main process >>>>> exited, code=killed, status=9/KILL >>>>> cze 01 10:36:12 raspberry-pi systemd[1]: weewx.service: Failed with >>>>> result 'signal'. >>>>> cze 01 10:36:12 raspberry-pi systemd[1]: weewx.service: Consumed 1h >>>>> 59min 21.575s CPU time. >>>>> >>>>> Yesterday at 17:52:56 there was the last entry to the database and >>>>> since then "nothing" has happened. Only today at 10:36:12 weewx stopped. >>>>> When I logged in. Yesterday I tried to log in around 18:00 and the system >>>>> was frozen. >>>>> While checking the logs, I found this information regarding weewx >>>>> version 5.0.2: >>>>> >>>>> WARNING weewx.engine: Previous report thread has been running >>>>> 601.9695854187012 seconds. Launching report thread anyway. >>>>> >>>>> INFO weewx.imagegenerator: Generated 13 images for report >>>>> SeasonsReport in 458.25 seconds >>>>> >>>>> INFO weewx.engine: Launch of report thread aborted: existing report >>>>> thread still running >>>>> >>>>> INFO weewx.cheetahgenerator: Generated 8 files for report >>>>> SeasonsReport in 252.10 seconds >>>>> >>>>> Generating simple images shouldn't take more than 30 seconds of time, >>>>> right? And not, as here, over 7 minutes in one case and 4 in the other. >>>>> In version 4.10.2, generating the same images takes: >>>>> >>>>> INFO weewx.cheetahgenerator: Generated 8 files for report >>>>> SeasonsReport in 12.51 seconds >>>>> >>>>> I came to the conclusion that this was no longer enough for my nerves >>>>> and I went back to version 4.10.2-1. Now the system works ok, nothing >>>>> loads >>>>> the processor, data is sent to weather services on an ongoing basis. The >>>>> local page is generated every two minutes. >>>>> In version 5.02. the page was refreshed at various intervals, when I >>>>> entered, I had data from 20 minutes ago. >>>>> >>>>> Below I am posting debug information from my settings for 5.02 and a >>>>> few drawings illustrating the problems. Maybe someone will be able to >>>>> find >>>>> out what is wrong in my configuration... Or maybe the problem is >>>>> somewhere >>>>> else... >>>>> >>>>> BR >>>>> Bartosz >>>>> >>>>> Debug file: https://gofile.me/75BgA/0SlVLSvVO >>>>> >>>>> CPU load 5.02 >>>>> [image: weewx.png] >>>>> >>>>> CPU load 4.10.2 >>>>> [image: weewx_4.10.2.png] >>>>> >>>>> db load 5.0.2 >>>>> [image: db.png] >>>>> >>>>> db load 4.10.2 >>>>> [image: db_4.10.2.png] >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "weewx-user" group. >>>>> >>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to weewx-user+...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/weewx-user/0b4d851a-98a6-4ec5-ab51-d7b17cca919an%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/weewx-user/0b4d851a-98a6-4ec5-ab51-d7b17cca919an%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to weewx-user+...@googlegroups.com. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/fd9b3506-a356-438c-8b9b-af93d0ae444fn%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/weewx-user/fd9b3506-a356-438c-8b9b-af93d0ae444fn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/375771a6-d705-4a62-9b86-4afa1e8803d7n%40googlegroups.com.
