Sounds fine, though what function you use to obtain archive record field values from the REST records depends on the observation; most will be ‘average’ but if rain is involved I expect it would be ‘sum’ not ‘average’, also if any status fields are involved they might be ‘last’ rather than ‘average’.
Thinking again regards archive interval. The driver needs to know the archive interval if it is to publish archive records either by backfill or normal generation. When the driver has an archive_interval property this knowledge is inherent, if this is no longer the case the driver needs to pick up the archive interval from the config dict and it needs to be able to handle the user changing the value. Gary On Tuesday, 9 February 2021 at 23:03:52 UTC+10 michael.k...@gmx.at wrote: > Thank you :) > > so when I said: > > "For backfilling missing data would use an approach like: If there is data > available in weatherflow REST service, an if it is available on a time base > which is more frequent than archive_interval, I would try to backfill like > this: for every top of an weewx archive_interval that is missing, I'd get > all data available for this interval, calculate averages (take care with > vector data windSpeed/windDir!), check if minimum/maximum values of each > REST entry, if it is lower/higher than the min/max value in > archive_day_[archiveColumn], handle it." > > I'd now say: > > For a weatherflow driver I wouldn't set an archive_interval in the driver > itself. For backfilling, get the all records from the REST within the > current archive_interval to backfill, create a single record for that > interval, with the values being the average values of all the REST records > (with the right treatment for vector data). Losing precision of min/max > values and their timestamps is acceptable (at least for reasonable short > archive_interval settings) > > > gjr80 schrieb am Dienstag, 9. Februar 2021 um 13:48:18 UTC+1: > >> Very simply, when your driver is emitting loop packets your time >> resolution is the time between loop packet timestamps, once your driver is >> emitting archive records your time resolution decreases and is the time >> between archive record timestamps. Similar but slightly different with >> actual values, loop packets give you the finest granularity of the >> observation values, once you move to archive records you get the average of >> the loop values (well actually you get whatever function is used to obtain >> an archive record value from a group of loop packet values, this is usually >> the average but it could be the sum or the last or the first value, it >> depends). >> >> Gary >> >> On Tuesday, 9 February 2021 at 22:42:18 UTC+10 gjr80 wrote: >> >>> OK, let's work through it. The hardware stored data at one minute >>> intervals, so between 11:05:00am to 11:10:00am the hardware will have five >>> stored records, the driver then creates a single archive record covering >>> 11:05:00am to 11:10:00am, the observation that has the max value at >>> 11:07:00am will appear as the average of the five values stored between >>> 11:05:00am and 11:10:00am (so chances are that will not be the same as the >>> value (the maximum) seen at 11:07:00am) and that is what will be stored as >>> the new daily max in the daily summary (if in fact it is a new maximum). >>> The timestamp of this backfilled record will be 11:10:00am and that is what >>> will be recorded as the max timestamp in the daily summary. So in this case >>> not only will you likely miss the maximum value that occurred at 11:07:00am >>> but you will miss the actual timestamp of the maximum value as well, it >>> will be moved to an archive interval boundary. >>> >>> Gary >>> >>> On Tuesday, 9 February 2021 at 22:28:52 UTC+10 michael.k...@gmx.at >>> wrote: >>> >>>> For the loop packets this is what I already thought to know. >>>> >>>> What I wanted to know if my weew archive interval is 5 mins, and the >>>> hardware stored at 1min: >>>> Lets assume the archive_interval 11:05:00am to 11:10:00am is to be >>>> backfilled. The driver gets the stored archive values from the hardware, >>>> where the maxvalue was at 11:07:00am. >>>> Is the maxvalue in the archive_day_xxx table for that day timestamped >>>> with 11:07:00am or with 11:10:00am? >>>> >>>> gjr80 schrieb am Dienstag, 9. Februar 2021 um 13:11:41 UTC+1: >>>> >>>>> If your one loop packet was the only loop packet received by WeeWX in >>>>> that archive interval then the archive record that WeeWX would synthesise >>>>> from that solitary loop packet would be the same as that loop packet. If >>>>> as >>>>> you said that loop packet was stored by hardware and if the driver's >>>>> genArchiveRecord() method obtains that same data when it generates an >>>>> archive record, then yes they will be the same. Highs and lows in the >>>>> daily >>>>> summaries would be the same but it would not surprise me if some >>>>> timestamps >>>>> were different. Loop packets are generally timestamped when the data is >>>>> read, whereas archive records are typically timestamped at the end of the >>>>> period they cover. So for an archive interval that covers say 11:05:00am >>>>> to >>>>> 11:06:00am, the loop packet may have been created at 11:05:25am and >>>>> timestamped accordingly. If WeeWX is set to obtain highs and lows from >>>>> loop >>>>> packets (ie loop_hilo >>>>> <http://weewx.com/docs/usersguide.htm#StdArchive> is True) then if a >>>>> new max/min is found in that loop packet the relevant timestamp recorded >>>>> in >>>>> the daily summary will be the loop packet timestamp. If you now consider >>>>> the archive record that is read during a backfill, it will be timestamped >>>>> 11:06:00am, again any new highs or lows will be recorded (so the actual >>>>> aggregate value will be the same) but in this case the timestamp recorded >>>>> for the high or low will be the archive record timestamp 11:06:00am. If >>>>> you >>>>> happen to have more than one loop packet in the archive period then you >>>>> may >>>>> loose some fidelity of the actual observation high/low value as well. >>>>> >>>>> This is the same effect you see if you rebuild your daily summaries >>>>> with wee_database --rebuild-daily, if you have loop_hilo = True then >>>>> your daily summary tables will likely have loop packet timestamps for >>>>> many >>>>> highs and lows but after you rebuild the daily summaries you loose that >>>>> fine detail and your high/low timestamps will align with archive record >>>>> timestamps. If you have multiple loop packets in an archive interval then >>>>> your high/low values may be one of the values in those loop packets, >>>>> after >>>>> rebuilding though you end up with archive values being used to generate >>>>> the >>>>> highs and lows which are typically averages of loop values. For example, >>>>> if >>>>> you have four loop packets in an archive interval and each has an >>>>> outTemp value of 8.0, 9.0, 12.0 and 7.0 degrees C your daily summary >>>>> may record 12.0 as the max. The corresponding archive record outTemp >>>>> value would (8.0+9.0+12.0+7.0)/4=9.0 so your max might be 9.0. >>>>> >>>>> Gary >>>>> >>>>> On Tuesday, 9 February 2021 at 21:03:10 UTC+10 michael.k...@gmx.at >>>>> wrote: >>>>> >>>>>> > If you mean does WeeWX determine if any incoming archive record has >>>>>> (for example) a new daily max for some field and then save that data, >>>>>> then >>>>>> yes that is what StdArchive does. >>>>>> >>>>>> So if a hardware sends a loop packet with all the readings in a >>>>>> one-minute-interval, and stores these exact values in its own storage in >>>>>> a >>>>>> separate entry, one entry per minute, the weewx database values are >>>>>> exactly >>>>>> the same, regardless if they were received as loop data as if they were >>>>>> backfilled? Including timestamps and Values for min/max in >>>>>> archive_day_xxxx, which may differ from the min/max value found in all >>>>>> weewx archive records of that particular day, with timestamps between >>>>>> possible timestamps of archve values? >>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-development+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/7a261cd7-7642-44bd-9a57-97bc7261b429n%40googlegroups.com.
