Ok, I see the issue now. Whilst changing wsum/sumtime worked in your case it only worked because you have a single constant archive interval in your database. Changing wsum/sumtime to sum/count in a database that has multiple different archive intervals would introduce errors. The correct fix (for all users) would have been to use wee_database with the —update option as per the WeeWX 3.7.0 upgrade instructions <http://weewx.com/docs/upgrading.htm#change_to_daily_summaries> when you upgraded to WeeWX v3.7.0 or beyond.
Also, to clarify your last point, you haven’t actually changed the values ‘to the way WeeWX calculates them’. Since WeeWX 3.7.0 the way WeeWX calculates wsum/sumtime is to weight them by the archive interval, your values are unweighted. Gary On Saturday, 28 November 2020 at 22:37:39 UTC+10 michael.k...@gmx.at wrote: > I guess the Problem is with the day of the version upgrade only. This day > produces values that are off. since the yearly average 2020 and the monthly > average 11/2020 is affected, both of these averages are wrong. 12/2020 > should have worked. > 11/01 til 11/13 worked also, so do 11/15 up until today 11/28. Only 11/14 > is wrong. So I guess fixing only all 11/14 values in all tables would have > fixed this for my particular case also. But hey, changing all the values to > the way weewx produces them currently won't hurt, I hope :) > > gjr80 schrieb am Samstag, 28. November 2020 um 13:23:28 UTC+1: > >> I’m not sure I see the problem you are trying to fix, I don’t see >> anything ‘obviously off’. The ‘weighting’ of archive records was reworked >> in WeeWX v3.7.0 to properly support multiple different archive intervals in >> a database. Your data appears to have a constant five minute archive >> interval, and in such a case your wsum and sumtime values are simply the >> sum and count fields multiplied by your (constant) archive interval of 300 >> seconds. Your average as returned by .avg (wsum/sumtime) will be identical >> to sum/count. If you are seeing the same situation as the OP I doubt the >> change in wsum/sumtime is the issue. >> >> Have you worked through the queries that Tom asked the OP to work >> through? Some solid numbers or other evidence of an error would help. >> >> Gary >> On Saturday, 28 November 2020 at 19:38:28 UTC+10 michael.k...@gmx.at >> wrote: >> >>> >>> 1606431600 -2.9 1606461229 6.5 1606483508 90.7493145743143 287 >>> 90.7493145743143 287 >>> 1606345200 -1.4 1606376580 5.8 1606396159 208.754783549784 288 >>> 208.754783549784 288 >>> 1606258800 -2.5 1606290557 5.3 1606310759 71.2823196248196 288 >>> 71.2823196248196 288 >>> 1606172400 -2.1 1606194900 9.9 1606224600 752.777460317461 286 >>> 752.777460317461 286 >>> 1606086000 0.29997 1606171800 7.6 1606136074 951.861259018759 288 >>> 951.861259018759 288 >>> 1605999600 -0.1001 1606073204 10.2 1606050999 884.716731601731 288 >>> 884.716731601731 288 >>> 1605913200 -1.6 1605940146 7.7 1605966081 442.098484848485 288 >>> 442.098484848485 288 >>> 1605826800 0.29997 1605913105 6.2 1605872440 1254.14992424242 287 >>> 1254.14992424242 287 >>> 1605740400 4.7 1605740431 13.9 1605793485 2373.86111111111 288 >>> 2373.86111111111 288 >>> 1605654000 3.1 1605733623 16.0 1605706154 2105.63747835498 288 >>> 2105.63747835498 288 >>> 1605567600 4.8 1605652944 13.6 1605615932 2268.75462121212 288 >>> 2268.75462121212 288 >>> 1605481200 1.4 1605493171 8.9 1605535929 1811.52088383838 288 >>> 1811.52088383838 288 >>> *1605394800 2.1 1605473931 16.1 1605445486 2346.52735930736 288 >>> 2346.52735930736 288* >>> *1605308400 5.5 1605328978 17.3 1605360608 2590.47363636364 288 >>> 650228.309292929 69058* >>> *1605222000 4.0 1605238302 14.9 1605272440 2255.38090909091 288 >>> 676614.272727273 86400* >>> 1605135600 2.7 1605160230 12.0 1605186834 1942.27053030303 288 >>> 582681.159090909 86400 >>> 1605049200 3.5 1605078802 11.3 1605095884 1937.71213564214 285 >>> 581313.640692641 85500 >>> 1604962800 1.2 1604988775 9.4 1605013839 1337.86883477633 287 >>> 401360.650466329 86100 >>> 1604876400 -0.1001 1604903428 9.3 1604925619 1065.95282828283 288 >>> 319785.848484848 86400 >>> >>> So I guess if you just set sumtime = count and wsum = sum for all >>> archive_day tables it should work? >>> michael.k...@gmx.at schrieb am Samstag, 28. November 2020 um 09:27:44 >>> UTC+1: >>> >>>> >>>> I am observing the same situation, as well as other WeeWX users near >>>> me. The average is clearly off since the 4.2.0 update. It also affects >>>> yearly average since then. So I guess this is something that happened with >>>> the 4.2.0 Version. I don't want to rebuild my daily values, how to find >>>> the >>>> bad values and correct them? Probably in the archive_daily table of the >>>> day >>>> I made the update? >>>> >>>> I found something: It's a change with "sum": >>>> 1-13 has "new" sumtime, 14 a mix (the day I updated) and old sumtime >>>> are 15 and and older: >>>> >>>> Isn't there a config that sets how this is calculated? >>>> >>>> >>>> b.cl...@gmail.com schrieb am Donnerstag, 19. November 2020 um 19:43:57 >>>> UTC+1: >>>> >>>>> Yeah, everything looks great again. >>>>> Thank you Tom for that excellent support. >>>>> Greetings from Suedlohn (Germany). >>>>> Berny >>>>> >>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um 17:56:56 >>>>> UTC+1: >>>>> >>>>>> So, for some reason, the weighted sum (field 'wsum') has too high a >>>>>> value, or the sum of observation time (field 'sumtime') has too low a >>>>>> value. >>>>>> >>>>>> The easiest fix is to just rebuild the daily summaries using the >>>>>> wee_database >>>>>> utility >>>>>> <http://www.weewx.com/docs/utilities.htm#wee_database_utility>. >>>>>> >>>>>> Stop weewxd. then, >>>>>> >>>>>> *wee_database --drop-daily* >>>>>> *wee_database --rebuild-daily* >>>>>> >>>>>> Restart weewxd >>>>>> >>>>>> For a database of your size, it shouldn't take more than a minute or >>>>>> two. >>>>>> >>>>>> It could take some time for the NOAA and html files to get corrected. >>>>>> You can speed things up by deleting them and allowing weewx to >>>>>> regenerate >>>>>> them. >>>>>> >>>>>> -tk >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Nov 19, 2020 at 8:48 AM Berny Cl <b.cl...@gmail.com> wrote: >>>>>> >>>>>>> sqlite> select sum(wsum)/sum(sumtime) from archive_day_outTemp where >>>>>>> strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>> 60.1308595259353 >>>>>>> >>>>>>> Ok, that looks the same like the value in my history table. >>>>>>> >>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>> 17:13:38 UTC+1: >>>>>>> >>>>>>>> 1. That looks reasonable. One other query to try: >>>>>>>> >>>>>>>> sqlite> *select sum(wsum)/sum(sumtime) from archive_day_outTemp >>>>>>>> where strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>> >>>>>>>> 2. If that doesn't reveal anything, I will send you an instrumented >>>>>>>> version of xtypes.py that will log the calculation. >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Nov 19, 2020 at 6:44 AM Berny Cl <b.cl...@gmail.com> wrote: >>>>>>>> >>>>>>>>> sqlite> select avg(outTemp) from archive where strftime("%Y-%m", >>>>>>>>> dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>> 51.117818676717 <(781)%20867-6717> >>>>>>>>> sqlite> select sum(sum)/sum(count) from archive_day_outTemp where >>>>>>>>> strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>> 51.114923603352 >>>>>>>>> >>>>>>>>> Thank you! >>>>>>>>> OK, I did that. The two numbers are very close. I think they are >>>>>>>>> correct (in Fahrenheit) but in my history table the temperature ist >>>>>>>>> too >>>>>>>>> high (in degree Celsius). >>>>>>>>> >>>>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>>> 15:02:55 UTC+1: >>>>>>>>> >>>>>>>>>> Most likely it's some bad data. Let's check the database >>>>>>>>>> directly. >>>>>>>>>> >>>>>>>>>> First find the database. If you did a package install, it's most >>>>>>>>>> likely at /var/lib/weewx/weewx.sdb. If you did a setup.py install, >>>>>>>>>> it's at >>>>>>>>>> /home/weewx/archive/weewx.sdb. Let's assume the former. >>>>>>>>>> >>>>>>>>>> Then, run two queries: >>>>>>>>>> >>>>>>>>>> *sqlite /var/lib/weewx/weewx.sdb* >>>>>>>>>> sqlite> *select avg(outTemp) from archive where >>>>>>>>>> strftime("%Y-%m", dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>> sqlite> *select sum(sum)/sum(count) from archive_day_outTemp >>>>>>>>>> where strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>> >>>>>>>>>> The first method calculates the average temperature for Nov 2020 >>>>>>>>>> by using the main archive table. The second by using the daily >>>>>>>>>> summaries. >>>>>>>>>> The two numbers should be very close. See what you get and we'll >>>>>>>>>> take >>>>>>>>>> it from there. >>>>>>>>>> >>>>>>>>>> -tk >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Nov 19, 2020 at 4:43 AM Berny Cl <b.cl...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi everybody, >>>>>>>>>>> since the last update to version 4.20, i have noticed an >>>>>>>>>>> incorrect value for the monthly average temperature at the history >>>>>>>>>>> table >>>>>>>>>>> and also in the monthly NOAA table. >>>>>>>>>>> I use the niculskin and my station is a FineOffset (WS 1080). >>>>>>>>>>> >>>>>>>>>>> See at: http://haus-volmering.de/history.html >>>>>>>>>>> (Durchschnittstemperatur = Average Temperature for Nov is >>>>>>>>>>> obviously incorrect) >>>>>>>>>>> >>>>>>>>>>> How can I fix that? >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/277ec811-21b4-41e9-8fc1-cda38d87014dn%40googlegroups.com >>>>>>>>>>> >>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/277ec811-21b4-41e9-8fc1-cda38d87014dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to weewx-user+...@googlegroups.com. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/weewx-user/87243ab4-47a4-4c85-b004-530f84e77673n%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/87243ab4-47a4-4c85-b004-530f84e77673n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "weewx-user" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to weewx-user+...@googlegroups.com. >>>>>>> >>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/weewx-user/f5a4768c-27f7-49c6-807e-496296d0c380n%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/weewx-user/f5a4768c-27f7-49c6-807e-496296d0c380n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/7f4bf885-e556-4946-bfcf-10ef89ad6964n%40googlegroups.com.
