I was able to restore all the data, doing as I planned: (i) the records collected during the power outage have been recovered from davis data logger, (ii) the subsequent records were already in the database but with wrong dateTime, so I just fixed its value shifting it 118m in the future. Then, as solid definitive solution for the future, I have also bought an RTC module for the rasp.
What is still puzzling me, is how is possible that the same wrong data (the wrong time value) was in the davis datalogger as well, for all the records in the period from after the power outage to the restoration of the correct time. Il giorno domenica 21 agosto 2022 alle 19:50:23 UTC+2 MikeQ ha scritto: > I have had a similar issue twice but not since January 2021. I have my > Davis console on wall power plus batteries, assuming the batteries would > work like a UPS but that might not be reliable. > > Look at the weewx wiki: > > https://github.com/weewx/weewx/wiki/Troubleshooting-the-Davis-Vantage-station#weewx-generates-html-pages-but-it-does-not-update-them > > I shut down weewx and inspected my database using DB Browser for SQLite on > a desktop and there are no issues in the DB so clock slew was not my issue. > > If I follow the instructions in 'Corrupt station memory' using debug mode > i see exactly the same results as the wiki shows with the DMPACT command > indicating the download is complete while I still have 197 pages in the > datalogger. > > Rebooting the console did not work for me and I had to use the dump & > clear commands. That resulted in the loss of records for a 25 minute > period the last time around. > > > > On Saturday, August 20, 2022 at 4:46:11 PM UTC-6 disaveri...@gmail.com > wrote: > >> Today I had a power outage, from the 06:01 CEST to the 07:51 CEST. >> So the last record added by weewx before the outage was at 06:01 CEST. >> From logs (reporting here only last two lines): >> >> pi@raspberrypi:~ $ journalctl -b -1 >> Aug 20 06:01:14 raspberrypi python3[16243]: weewx[16243] INFO >> weewx.manager: Added record 2022-08-20 06:01:00 CEST (1660968060) to >> database 'weewx.sdb' >> Aug 20 06:01:14 raspberrypi python3[16243]: weewx[16243] INFO >> weewx.manager: Added record 2022-08-20 06:01:00 CEST (1660968060) to daily >> summary in 'weewx.s> >> >> When the power came back, the wifi was not available, so the raspberry's >> time (I have no RTC module) was automatically set to a previously recorded >> timestamp: 05:53:32 CEST (118 mins before real time). >> So when weewx re-started recording, it discarded first 8 samples and then >> started going. From logs: >> >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 05:54:00 CEST (1660967640) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 05:55:00 CEST (1660967700) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 05:56:00 CEST (1660967760) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 05:57:00 CEST (1660967820) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 05:58:00 CEST (1660967880) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 05:59:00 CEST (1660967940) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 06:00:00 CEST (1660968000) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:01:19 raspberrypi python3[600]: weewx[600] ERROR >> weewx.manager: Unable to add record 2022-08-20 06:01:00 CEST (1660968060) >> to database 'weewx.sdb': UNIQUE constraint failed: archive.dateTime >> Aug 20 06:02:16 raspberrypi python3[600]: weewx[600] INFO >> weewx.manager: Added record 2022-08-20 06:02:00 CEST (1660968120) to >> database 'weewx.sdb' >> Aug 20 06:02:16 raspberrypi python3[600]: weewx[600] INFO >> weewx.manager: Added record 2022-08-20 06:02:00 CEST (1660968120) to daily >> summary in 'weewx.sdb' >> >> But, and this is a bit surprising, it also discarded records collected in >> the davis datalogger. >> They are 110 lines, from 1660968120 (06:02:00 CEST) and 1660974660 >> (07:51:00 CEST) (The power outage didn't affect the davis station: it is >> battery powered). >> From logs (reporting only the first and the last one): >> >> Aug 20 05:53:35 raspberrypi python3[600]: weewx[600] WARNING >> weewx.engine: Ignore historical record: {'dateTime': 1660968120, 'usUnits': >> 1, 'interval': 1, 'rxCheckPercent': 89.6875, 'outTemp': 68.8> >> ... >> Aug 20 05:53:35 raspberrypi python3[600]: weewx[600] WARNING >> weewx.engine: Ignore historical record: {'dateTime': 1660974660, 'usUnits': >> 1, 'interval': 1, 'rxCheckPercent': 100.0, 'outTemp': 69.5, > >> >> Why were they discarded? Because the weewx time was before the record's >> time? >> >> Then, few hours later I restored the wifi network, and the rasp >> automatically changed his time, and so weewx. From logs: >> >> Aug 20 12:09:33 raspberrypi systemd[1]: Started Network Time >> Synchronization. >> Aug 20 14:07:04 raspberrypi systemd-timesyncd[2057]: Initial >> synchronization to time server 93.94.88.51:123 (0.debian.pool.ntp.org). >> Aug 20 14:07:05 raspberrypi python3[600]: weewx[600] INFO >> weewx.engine: Clock error is -7050.52 seconds (positive is fast) >> Aug 20 14:07:05 raspberrypi python3[600]: weewx[600] INFO >> weewx.drivers.vantage: Clock set to 2022-08-20 14:07:06 CEST (1660997226) >> >> After this, it started to store data correctly, starting from 1660997280 >> (14:08:00 CEST). >> >> Now the situation is: >> - I have a hole from the 1660990140 (12:09:00 CEST) to 1660997280 >> (14:08:00 CEST) (excluded). (When I restored the network connection, and >> rasp fixed its time). >> - All the database records with dateTime from 1660968120 (06:02:00 CEST) >> to 1660990140 (12:09:00 CEST) (included) are backshifted of 118 minutes >> - Real records from 1660968120 (06:02:00 CEST) to 1660974660 (07:51:00 >> CEST) are in the datalogger so I can restore them. >> - The only records I lost, are the first 8 samples weewx took after >> restart (from 07:52:00 CEST to 07:59:00 CEST, included) >> >> My plan to fix data: >> 1. drop (but preserving for later use) all the records starting from >> 1660968120 (06:02:00 CEST) (included) from the `archive` table in the >> database >> 2. start weewx with datalogger activated to download records from >> 1660968120 (06:02:00 CEST) to 1660974660 (07:51:00 CEST) >> 3. re-insert records I removed at step 1. insert only from 1660968120 >> (06:02:00 CEST) to 1660990140 (12:09:00 CEST), adding 118*60 to dateTime >> 4. re-insert all the following records (so starting from 1660997280 >> (14:08:00 CEST)) keeping them as they are >> 5. fix other table running `wee_database --drop-daily` and then >> `wee_database --rebuild-daily` (maybe scoped to the single day of today) >> >> Any suggestion, or comment? >> >> Thanks >> >> >> [But at the end, I have a question: Why in the datalogger, records >> starting from 07:52:00 CEST, have the same wrong values present in the >> weewx database? Is the datalogger written by weewx? >> I mean: It looks like the datalogger has been written by weewx when (i) >> no data were present AND (ii) weewx was connected to him (so, starting from >> 07:52:00 CEST. before data are present, collected by the station itself)] >> >> -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/222c7edc-6798-4514-bbcd-44a33155d9dcn%40googlegroups.com.
