Thanks. Somehow, your daily summary shifted from V1.0 to V2.0 behavior starting with the 1605308400 timestamp.
Gary: the only thing I can think of that changed in V4.2 is this commit <https://github.com/weewx/weewx/commit/12c635aabaae7be7181006c7164a336e917faade>, which was in response to this thread <https://groups.google.com/g/weewx-user/c/hRbR5p4A9BM/m/sPHi4iSSBwAJ>. Offhand, I can't see how the commit would cause Michael's symptoms. On Sat, Nov 28, 2020 at 6:13 AM michael.k...@gmx.at < michael.kainzba...@gmx.at> wrote: > Version 2.0 > > tke...@gmail.com schrieb am Samstag, 28. November 2020 um 15:00:04 UTC+1: > >> Michael, >> >> Can you check what the version number is for your daily summaries? >> >> *select * from archive_day__metadata;* >> >> should do it. >> >> -tk >> >> On Sat, Nov 28, 2020 at 5:21 AM michael.k...@gmx.at <michael.k...@gmx.at> >> wrote: >> >>> >>> I got the point, with the constant archive interval. But why did it >>> change from weighted wsum and sumtime values to unweighted when upgrading >>> 4.1.1 => 4.2.0? Did I unintentionally change a config portion when merging >>> the configs? I guess many users aren't aware that this happened to them >>> also. 3.7.0 was released in March 2017, my friend who is having the same >>> issue started with 3.8.2 in Nov. 2018. >>> gjr80 schrieb am Samstag, 28. November 2020 um 13:58:24 UTC+1: >>> >>>> Ok, I see the issue now. >>>> >>>> Whilst changing wsum/sumtime worked in your case it only worked because >>>> you have a single constant archive interval in your database. Changing >>>> wsum/sumtime to sum/count in a database that has multiple different archive >>>> intervals would introduce errors. The correct fix (for all users) would >>>> have been to use wee_database with the —update option as per the >>>> WeeWX 3.7.0 upgrade instructions >>>> <http://weewx.com/docs/upgrading.htm#change_to_daily_summaries> when >>>> you upgraded to WeeWX v3.7.0 or beyond. >>>> >>>> Also, to clarify your last point, you haven’t actually changed the >>>> values ‘to the way WeeWX calculates them’. Since WeeWX 3.7.0 the way WeeWX >>>> calculates wsum/sumtime is to weight them by the archive interval, your >>>> values are unweighted. >>>> >>>> Gary >>>> >>>> On Saturday, 28 November 2020 at 22:37:39 UTC+10 michael.k...@gmx.at >>>> wrote: >>>> >>>>> I guess the Problem is with the day of the version upgrade only. This >>>>> day produces values that are off. since the yearly average 2020 and the >>>>> monthly average 11/2020 is affected, both of these averages are wrong. >>>>> 12/2020 should have worked. >>>>> 11/01 til 11/13 worked also, so do 11/15 up until today 11/28. Only >>>>> 11/14 is wrong. So I guess fixing only all 11/14 values in all tables >>>>> would >>>>> have fixed this for my particular case also. But hey, changing all the >>>>> values to the way weewx produces them currently won't hurt, I hope :) >>>>> >>>>> gjr80 schrieb am Samstag, 28. November 2020 um 13:23:28 UTC+1: >>>>> >>>>>> I’m not sure I see the problem you are trying to fix, I don’t see >>>>>> anything ‘obviously off’. The ‘weighting’ of archive records was reworked >>>>>> in WeeWX v3.7.0 to properly support multiple different archive intervals >>>>>> in >>>>>> a database. Your data appears to have a constant five minute archive >>>>>> interval, and in such a case your wsum and sumtime values are simply the >>>>>> sum and count fields multiplied by your (constant) archive interval of >>>>>> 300 >>>>>> seconds. Your average as returned by .avg (wsum/sumtime) will be >>>>>> identical >>>>>> to sum/count. If you are seeing the same situation as the OP I doubt the >>>>>> change in wsum/sumtime is the issue. >>>>>> >>>>>> Have you worked through the queries that Tom asked the OP to work >>>>>> through? Some solid numbers or other evidence of an error would help. >>>>>> >>>>>> Gary >>>>>> On Saturday, 28 November 2020 at 19:38:28 UTC+10 michael.k...@gmx.at >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> 1606431600 -2.9 1606461229 6.5 1606483508 90.7493145743143 287 >>>>>>> 90.7493145743143 287 >>>>>>> 1606345200 -1.4 1606376580 5.8 1606396159 208.754783549784 288 >>>>>>> 208.754783549784 288 >>>>>>> 1606258800 -2.5 1606290557 5.3 1606310759 71.2823196248196 288 >>>>>>> 71.2823196248196 288 >>>>>>> 1606172400 -2.1 1606194900 9.9 1606224600 752.777460317461 286 >>>>>>> 752.777460317461 286 >>>>>>> 1606086000 0.29997 1606171800 7.6 1606136074 951.861259018759 288 >>>>>>> 951.861259018759 288 >>>>>>> 1605999600 -0.1001 1606073204 10.2 1606050999 884.716731601731 288 >>>>>>> 884.716731601731 288 >>>>>>> 1605913200 -1.6 1605940146 7.7 1605966081 442.098484848485 288 >>>>>>> 442.098484848485 288 >>>>>>> 1605826800 0.29997 1605913105 6.2 1605872440 1254.14992424242 287 >>>>>>> 1254.14992424242 287 >>>>>>> 1605740400 4.7 1605740431 13.9 1605793485 2373.86111111111 288 >>>>>>> 2373.86111111111 288 >>>>>>> 1605654000 3.1 1605733623 16.0 1605706154 2105.63747835498 288 >>>>>>> 2105.63747835498 288 >>>>>>> 1605567600 4.8 1605652944 13.6 1605615932 2268.75462121212 288 >>>>>>> 2268.75462121212 288 >>>>>>> 1605481200 1.4 1605493171 8.9 1605535929 1811.52088383838 288 >>>>>>> 1811.52088383838 288 >>>>>>> *1605394800 2.1 1605473931 16.1 1605445486 2346.52735930736 288 >>>>>>> 2346.52735930736 288* >>>>>>> *1605308400 5.5 1605328978 17.3 1605360608 2590.47363636364 288 >>>>>>> 650228.309292929 69058* >>>>>>> *1605222000 4.0 1605238302 14.9 1605272440 2255.38090909091 288 >>>>>>> 676614.272727273 86400* >>>>>>> 1605135600 2.7 1605160230 12.0 1605186834 1942.27053030303 288 >>>>>>> 582681.159090909 86400 >>>>>>> 1605049200 3.5 1605078802 11.3 1605095884 1937.71213564214 285 >>>>>>> 581313.640692641 85500 >>>>>>> 1604962800 1.2 1604988775 9.4 1605013839 1337.86883477633 287 >>>>>>> 401360.650466329 86100 >>>>>>> 1604876400 -0.1001 1604903428 9.3 1604925619 1065.95282828283 288 >>>>>>> 319785.848484848 86400 >>>>>>> >>>>>>> So I guess if you just set sumtime = count and wsum = sum for all >>>>>>> archive_day tables it should work? >>>>>>> michael.k...@gmx.at schrieb am Samstag, 28. November 2020 um >>>>>>> 09:27:44 UTC+1: >>>>>>> >>>>>>>> >>>>>>>> I am observing the same situation, as well as other WeeWX users >>>>>>>> near me. The average is clearly off since the 4.2.0 update. It also >>>>>>>> affects >>>>>>>> yearly average since then. So I guess this is something that happened >>>>>>>> with >>>>>>>> the 4.2.0 Version. I don't want to rebuild my daily values, how to >>>>>>>> find the >>>>>>>> bad values and correct them? Probably in the archive_daily table of >>>>>>>> the day >>>>>>>> I made the update? >>>>>>>> >>>>>>>> I found something: It's a change with "sum": >>>>>>>> 1-13 has "new" sumtime, 14 a mix (the day I updated) and old >>>>>>>> sumtime are 15 and and older: >>>>>>>> >>>>>>>> Isn't there a config that sets how this is calculated? >>>>>>>> >>>>>>>> >>>>>>>> b.cl...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>> 19:43:57 UTC+1: >>>>>>>> >>>>>>>>> Yeah, everything looks great again. >>>>>>>>> Thank you Tom for that excellent support. >>>>>>>>> Greetings from Suedlohn (Germany). >>>>>>>>> Berny >>>>>>>>> >>>>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>>> 17:56:56 UTC+1: >>>>>>>>> >>>>>>>>>> So, for some reason, the weighted sum (field 'wsum') has too high >>>>>>>>>> a value, or the sum of observation time (field 'sumtime') has too >>>>>>>>>> low a >>>>>>>>>> value. >>>>>>>>>> >>>>>>>>>> The easiest fix is to just rebuild the daily summaries using the >>>>>>>>>> wee_database >>>>>>>>>> utility >>>>>>>>>> <http://www.weewx.com/docs/utilities.htm#wee_database_utility>. >>>>>>>>>> >>>>>>>>>> Stop weewxd. then, >>>>>>>>>> >>>>>>>>>> *wee_database --drop-daily* >>>>>>>>>> *wee_database --rebuild-daily* >>>>>>>>>> >>>>>>>>>> Restart weewxd >>>>>>>>>> >>>>>>>>>> For a database of your size, it shouldn't take more than a minute >>>>>>>>>> or two. >>>>>>>>>> >>>>>>>>>> It could take some time for the NOAA and html files to get >>>>>>>>>> corrected. You can speed things up by deleting them and allowing >>>>>>>>>> weewx to >>>>>>>>>> regenerate them. >>>>>>>>>> >>>>>>>>>> -tk >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Nov 19, 2020 at 8:48 AM Berny Cl <b.cl...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> sqlite> select sum(wsum)/sum(sumtime) from archive_day_outTemp >>>>>>>>>>> where strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>>>> 60.1308595259353 >>>>>>>>>>> >>>>>>>>>>> Ok, that looks the same like the value in my history table. >>>>>>>>>>> >>>>>>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>>>>> 17:13:38 UTC+1: >>>>>>>>>>> >>>>>>>>>>>> 1. That looks reasonable. One other query to try: >>>>>>>>>>>> >>>>>>>>>>>> sqlite> *select sum(wsum)/sum(sumtime) from >>>>>>>>>>>> archive_day_outTemp where >>>>>>>>>>>> strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>>>> >>>>>>>>>>>> 2. If that doesn't reveal anything, I will send you an >>>>>>>>>>>> instrumented version of xtypes.py that will log the calculation. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 19, 2020 at 6:44 AM Berny Cl <b.cl...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> sqlite> select avg(outTemp) from archive where >>>>>>>>>>>>> strftime("%Y-%m", dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>>>>>> 51.117818676717 <(781)%20867-6717> >>>>>>>>>>>>> sqlite> select sum(sum)/sum(count) from archive_day_outTemp >>>>>>>>>>>>> where >>>>>>>>>>>>> strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>>>>>> 51.114923603352 >>>>>>>>>>>>> >>>>>>>>>>>>> Thank you! >>>>>>>>>>>>> OK, I did that. The two numbers are very close. I think they >>>>>>>>>>>>> are correct (in Fahrenheit) but in my history table the >>>>>>>>>>>>> temperature ist too >>>>>>>>>>>>> high (in degree Celsius). >>>>>>>>>>>>> >>>>>>>>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>>>>>>> 15:02:55 UTC+1: >>>>>>>>>>>>> >>>>>>>>>>>>>> Most likely it's some bad data. Let's check the database >>>>>>>>>>>>>> directly. >>>>>>>>>>>>>> >>>>>>>>>>>>>> First find the database. If you did a package install, it's >>>>>>>>>>>>>> most likely at /var/lib/weewx/weewx.sdb. If you did a setup.py >>>>>>>>>>>>>> install, >>>>>>>>>>>>>> it's at /home/weewx/archive/weewx.sdb. Let's assume the former. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Then, run two queries: >>>>>>>>>>>>>> >>>>>>>>>>>>>> *sqlite /var/lib/weewx/weewx.sdb* >>>>>>>>>>>>>> sqlite> *select avg(outTemp) from archive where >>>>>>>>>>>>>> strftime("%Y-%m", dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>>>>>> sqlite> *select sum(sum)/sum(count) from archive_day_outTemp >>>>>>>>>>>>>> where >>>>>>>>>>>>>> strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>>>>>> >>>>>>>>>>>>>> The first method calculates the average temperature for Nov >>>>>>>>>>>>>> 2020 by using the main archive table. The second by using the >>>>>>>>>>>>>> daily >>>>>>>>>>>>>> summaries. The two numbers should be very close. See what you >>>>>>>>>>>>>> get and we'll >>>>>>>>>>>>>> take it from there. >>>>>>>>>>>>>> >>>>>>>>>>>>>> -tk >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, Nov 19, 2020 at 4:43 AM Berny Cl <b.cl...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi everybody, >>>>>>>>>>>>>>> since the last update to version 4.20, i have noticed an >>>>>>>>>>>>>>> incorrect value for the monthly average temperature at the >>>>>>>>>>>>>>> history table >>>>>>>>>>>>>>> and also in the monthly NOAA table. >>>>>>>>>>>>>>> I use the niculskin and my station is a FineOffset (WS 1080). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> See at: http://haus-volmering.de/history.html >>>>>>>>>>>>>>> (Durchschnittstemperatur = Average Temperature for Nov is >>>>>>>>>>>>>>> obviously incorrect) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> How can I fix that? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails >>>>>>>>>>>>>>> from it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/277ec811-21b4-41e9-8fc1-cda38d87014dn%40googlegroups.com >>>>>>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/277ec811-21b4-41e9-8fc1-cda38d87014dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>>>> . >>>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>>>> >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/87243ab4-47a4-4c85-b004-530f84e77673n%40googlegroups.com >>>>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/87243ab4-47a4-4c85-b004-530f84e77673n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/f5a4768c-27f7-49c6-807e-496296d0c380n%40googlegroups.com >>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/f5a4768c-27f7-49c6-807e-496296d0c380n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to weewx-user+...@googlegroups.com. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/73ded510-7efd-49b4-b2b5-00d6cf9ff806n%40googlegroups.com >>> <https://groups.google.com/d/msgid/weewx-user/73ded510-7efd-49b4-b2b5-00d6cf9ff806n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to weewx-user+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/ad81bb6f-896e-4236-ac9d-49ed688ad04dn%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/ad81bb6f-896e-4236-ac9d-49ed688ad04dn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zED%3Ds0BfW8Jrhe4kiQMVziQeLqGMo4MqbCboN1nVCJR3tg%40mail.gmail.com.
