Michael, Can you check what the version number is for your daily summaries?
*select * from archive_day__metadata;* should do it. -tk On Sat, Nov 28, 2020 at 5:21 AM michael.k...@gmx.at < michael.kainzba...@gmx.at> wrote: > > I got the point, with the constant archive interval. But why did it change > from weighted wsum and sumtime values to unweighted when upgrading 4.1.1 => > 4.2.0? Did I unintentionally change a config portion when merging the > configs? I guess many users aren't aware that this happened to them also. > 3.7.0 was released in March 2017, my friend who is having the same issue > started with 3.8.2 in Nov. 2018. > gjr80 schrieb am Samstag, 28. November 2020 um 13:58:24 UTC+1: > >> Ok, I see the issue now. >> >> Whilst changing wsum/sumtime worked in your case it only worked because >> you have a single constant archive interval in your database. Changing >> wsum/sumtime to sum/count in a database that has multiple different archive >> intervals would introduce errors. The correct fix (for all users) would >> have been to use wee_database with the —update option as per the WeeWX >> 3.7.0 upgrade instructions >> <http://weewx.com/docs/upgrading.htm#change_to_daily_summaries> when you >> upgraded to WeeWX v3.7.0 or beyond. >> >> Also, to clarify your last point, you haven’t actually changed the values >> ‘to the way WeeWX calculates them’. Since WeeWX 3.7.0 the way WeeWX >> calculates wsum/sumtime is to weight them by the archive interval, your >> values are unweighted. >> >> Gary >> >> On Saturday, 28 November 2020 at 22:37:39 UTC+10 michael.k...@gmx.at >> wrote: >> >>> I guess the Problem is with the day of the version upgrade only. This >>> day produces values that are off. since the yearly average 2020 and the >>> monthly average 11/2020 is affected, both of these averages are wrong. >>> 12/2020 should have worked. >>> 11/01 til 11/13 worked also, so do 11/15 up until today 11/28. Only >>> 11/14 is wrong. So I guess fixing only all 11/14 values in all tables would >>> have fixed this for my particular case also. But hey, changing all the >>> values to the way weewx produces them currently won't hurt, I hope :) >>> >>> gjr80 schrieb am Samstag, 28. November 2020 um 13:23:28 UTC+1: >>> >>>> I’m not sure I see the problem you are trying to fix, I don’t see >>>> anything ‘obviously off’. The ‘weighting’ of archive records was reworked >>>> in WeeWX v3.7.0 to properly support multiple different archive intervals in >>>> a database. Your data appears to have a constant five minute archive >>>> interval, and in such a case your wsum and sumtime values are simply the >>>> sum and count fields multiplied by your (constant) archive interval of 300 >>>> seconds. Your average as returned by .avg (wsum/sumtime) will be identical >>>> to sum/count. If you are seeing the same situation as the OP I doubt the >>>> change in wsum/sumtime is the issue. >>>> >>>> Have you worked through the queries that Tom asked the OP to work >>>> through? Some solid numbers or other evidence of an error would help. >>>> >>>> Gary >>>> On Saturday, 28 November 2020 at 19:38:28 UTC+10 michael.k...@gmx.at >>>> wrote: >>>> >>>>> >>>>> 1606431600 -2.9 1606461229 6.5 1606483508 90.7493145743143 287 >>>>> 90.7493145743143 287 >>>>> 1606345200 -1.4 1606376580 5.8 1606396159 208.754783549784 288 >>>>> 208.754783549784 288 >>>>> 1606258800 -2.5 1606290557 5.3 1606310759 71.2823196248196 288 >>>>> 71.2823196248196 288 >>>>> 1606172400 -2.1 1606194900 9.9 1606224600 752.777460317461 286 >>>>> 752.777460317461 286 >>>>> 1606086000 0.29997 1606171800 7.6 1606136074 951.861259018759 288 >>>>> 951.861259018759 288 >>>>> 1605999600 -0.1001 1606073204 10.2 1606050999 884.716731601731 288 >>>>> 884.716731601731 288 >>>>> 1605913200 -1.6 1605940146 7.7 1605966081 442.098484848485 288 >>>>> 442.098484848485 288 >>>>> 1605826800 0.29997 1605913105 6.2 1605872440 1254.14992424242 287 >>>>> 1254.14992424242 287 >>>>> 1605740400 4.7 1605740431 13.9 1605793485 2373.86111111111 288 >>>>> 2373.86111111111 288 >>>>> 1605654000 3.1 1605733623 16.0 1605706154 2105.63747835498 288 >>>>> 2105.63747835498 288 >>>>> 1605567600 4.8 1605652944 13.6 1605615932 2268.75462121212 288 >>>>> 2268.75462121212 288 >>>>> 1605481200 1.4 1605493171 8.9 1605535929 1811.52088383838 288 >>>>> 1811.52088383838 288 >>>>> *1605394800 2.1 1605473931 16.1 1605445486 2346.52735930736 288 >>>>> 2346.52735930736 288* >>>>> *1605308400 5.5 1605328978 17.3 1605360608 2590.47363636364 288 >>>>> 650228.309292929 69058* >>>>> *1605222000 4.0 1605238302 14.9 1605272440 2255.38090909091 288 >>>>> 676614.272727273 86400* >>>>> 1605135600 2.7 1605160230 12.0 1605186834 1942.27053030303 288 >>>>> 582681.159090909 86400 >>>>> 1605049200 3.5 1605078802 11.3 1605095884 1937.71213564214 285 >>>>> 581313.640692641 85500 >>>>> 1604962800 1.2 1604988775 9.4 1605013839 1337.86883477633 287 >>>>> 401360.650466329 86100 >>>>> 1604876400 -0.1001 1604903428 9.3 1604925619 1065.95282828283 288 >>>>> 319785.848484848 86400 >>>>> >>>>> So I guess if you just set sumtime = count and wsum = sum for all >>>>> archive_day tables it should work? >>>>> michael.k...@gmx.at schrieb am Samstag, 28. November 2020 um 09:27:44 >>>>> UTC+1: >>>>> >>>>>> >>>>>> I am observing the same situation, as well as other WeeWX users near >>>>>> me. The average is clearly off since the 4.2.0 update. It also affects >>>>>> yearly average since then. So I guess this is something that happened >>>>>> with >>>>>> the 4.2.0 Version. I don't want to rebuild my daily values, how to find >>>>>> the >>>>>> bad values and correct them? Probably in the archive_daily table of the >>>>>> day >>>>>> I made the update? >>>>>> >>>>>> I found something: It's a change with "sum": >>>>>> 1-13 has "new" sumtime, 14 a mix (the day I updated) and old sumtime >>>>>> are 15 and and older: >>>>>> >>>>>> Isn't there a config that sets how this is calculated? >>>>>> >>>>>> >>>>>> b.cl...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>> 19:43:57 UTC+1: >>>>>> >>>>>>> Yeah, everything looks great again. >>>>>>> Thank you Tom for that excellent support. >>>>>>> Greetings from Suedlohn (Germany). >>>>>>> Berny >>>>>>> >>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>> 17:56:56 UTC+1: >>>>>>> >>>>>>>> So, for some reason, the weighted sum (field 'wsum') has too high a >>>>>>>> value, or the sum of observation time (field 'sumtime') has too low a >>>>>>>> value. >>>>>>>> >>>>>>>> The easiest fix is to just rebuild the daily summaries using the >>>>>>>> wee_database >>>>>>>> utility >>>>>>>> <http://www.weewx.com/docs/utilities.htm#wee_database_utility>. >>>>>>>> >>>>>>>> Stop weewxd. then, >>>>>>>> >>>>>>>> *wee_database --drop-daily* >>>>>>>> *wee_database --rebuild-daily* >>>>>>>> >>>>>>>> Restart weewxd >>>>>>>> >>>>>>>> For a database of your size, it shouldn't take more than a minute >>>>>>>> or two. >>>>>>>> >>>>>>>> It could take some time for the NOAA and html files to get >>>>>>>> corrected. You can speed things up by deleting them and allowing weewx >>>>>>>> to >>>>>>>> regenerate them. >>>>>>>> >>>>>>>> -tk >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Nov 19, 2020 at 8:48 AM Berny Cl <b.cl...@gmail.com> wrote: >>>>>>>> >>>>>>>>> sqlite> select sum(wsum)/sum(sumtime) from archive_day_outTemp >>>>>>>>> where strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>> 60.1308595259353 >>>>>>>>> >>>>>>>>> Ok, that looks the same like the value in my history table. >>>>>>>>> >>>>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>>> 17:13:38 UTC+1: >>>>>>>>> >>>>>>>>>> 1. That looks reasonable. One other query to try: >>>>>>>>>> >>>>>>>>>> sqlite> *select sum(wsum)/sum(sumtime) from archive_day_outTemp >>>>>>>>>> where strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>> >>>>>>>>>> 2. If that doesn't reveal anything, I will send you an >>>>>>>>>> instrumented version of xtypes.py that will log the calculation. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, Nov 19, 2020 at 6:44 AM Berny Cl <b.cl...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> sqlite> select avg(outTemp) from archive where strftime("%Y-%m", >>>>>>>>>>> dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>>>> 51.117818676717 <(781)%20867-6717> >>>>>>>>>>> sqlite> select sum(sum)/sum(count) from archive_day_outTemp >>>>>>>>>>> where strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11'; >>>>>>>>>>> 51.114923603352 >>>>>>>>>>> >>>>>>>>>>> Thank you! >>>>>>>>>>> OK, I did that. The two numbers are very close. I think they are >>>>>>>>>>> correct (in Fahrenheit) but in my history table the temperature ist >>>>>>>>>>> too >>>>>>>>>>> high (in degree Celsius). >>>>>>>>>>> >>>>>>>>>>> tke...@gmail.com schrieb am Donnerstag, 19. November 2020 um >>>>>>>>>>> 15:02:55 UTC+1: >>>>>>>>>>> >>>>>>>>>>>> Most likely it's some bad data. Let's check the database >>>>>>>>>>>> directly. >>>>>>>>>>>> >>>>>>>>>>>> First find the database. If you did a package install, it's >>>>>>>>>>>> most likely at /var/lib/weewx/weewx.sdb. If you did a setup.py >>>>>>>>>>>> install, >>>>>>>>>>>> it's at /home/weewx/archive/weewx.sdb. Let's assume the former. >>>>>>>>>>>> >>>>>>>>>>>> Then, run two queries: >>>>>>>>>>>> >>>>>>>>>>>> *sqlite /var/lib/weewx/weewx.sdb* >>>>>>>>>>>> sqlite> *select avg(outTemp) from archive where >>>>>>>>>>>> strftime("%Y-%m", dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>>>> sqlite> *select sum(sum)/sum(count) from archive_day_outTemp >>>>>>>>>>>> where >>>>>>>>>>>> strftime("%Y-%m",dateTime,'unixepoch','localtime')=='2020-11';* >>>>>>>>>>>> >>>>>>>>>>>> The first method calculates the average temperature for Nov >>>>>>>>>>>> 2020 by using the main archive table. The second by using the daily >>>>>>>>>>>> summaries. The two numbers should be very close. See what you get >>>>>>>>>>>> and we'll >>>>>>>>>>>> take it from there. >>>>>>>>>>>> >>>>>>>>>>>> -tk >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 19, 2020 at 4:43 AM Berny Cl <b.cl...@gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi everybody, >>>>>>>>>>>>> since the last update to version 4.20, i have noticed an >>>>>>>>>>>>> incorrect value for the monthly average temperature at the >>>>>>>>>>>>> history table >>>>>>>>>>>>> and also in the monthly NOAA table. >>>>>>>>>>>>> I use the niculskin and my station is a FineOffset (WS 1080). >>>>>>>>>>>>> >>>>>>>>>>>>> See at: http://haus-volmering.de/history.html >>>>>>>>>>>>> (Durchschnittstemperatur = Average Temperature for Nov is >>>>>>>>>>>>> obviously incorrect) >>>>>>>>>>>>> >>>>>>>>>>>>> How can I fix that? >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>>>> it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/277ec811-21b4-41e9-8fc1-cda38d87014dn%40googlegroups.com >>>>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/277ec811-21b4-41e9-8fc1-cda38d87014dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>>>> . >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>> Google Groups "weewx-user" group. >>>>>>>>>>> To unsubscribe from this group and stop receiving emails from >>>>>>>>>>> it, send an email to weewx-user+...@googlegroups.com. >>>>>>>>>>> >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>> https://groups.google.com/d/msgid/weewx-user/87243ab4-47a4-4c85-b004-530f84e77673n%40googlegroups.com >>>>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/87243ab4-47a4-4c85-b004-530f84e77673n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "weewx-user" group. >>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to weewx-user+...@googlegroups.com. >>>>>>>>> >>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/weewx-user/f5a4768c-27f7-49c6-807e-496296d0c380n%40googlegroups.com >>>>>>>>> <https://groups.google.com/d/msgid/weewx-user/f5a4768c-27f7-49c6-807e-496296d0c380n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> >>>>>>>> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to weewx-user+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/73ded510-7efd-49b4-b2b5-00d6cf9ff806n%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/73ded510-7efd-49b4-b2b5-00d6cf9ff806n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEBmvx2nbTRaLyg5%2Bf7p4KhCNekBdR67WdLB6kzQBmLOAg%40mail.gmail.com.
