[weewx-user] Re: Interesting wunderfixer behavior
If I were you I would not try and understand the foibles of WU - it is just not worth the effort. Remember - WU does not exist to provide a free cloud backup data repository for global weather stations around the world!! If you keep hammering the service with 10 minute wunderfixer runs they are more likely to just remove the api facility totally than make it able to cope with higher demands!!! OWM now appears to be suffering as I suspect many users have moved from WU to OWM . On Saturday, 4 May 2019 20:54:55 UTC+3, Leon Shaner wrote: > > Hey, WeeWX'ers! > > TL:DR, scroll to the bottom for the question and scroll back as you need. > ;-) > > I posted last week about WU "capacity" issues and the need to increase > wunderfixer timeout from 10 seconds to 20 seconds to compensate. That's > working great, so far. > > This morning my RPI -> WMR300 comms went south due to the well-known, but > as of yet unfixed issue somewhere between the Kernel/USB handling and the > flakey firmware on the WMR300. :-/ > > My weewx_watchdog script went through all the paces that I designed it to > do and all there worked perfectly. After my watchdog tried the weewx > restart (which wasn't enough [never is in my case]), then after another 10 > minutes passed, it did the reboot, and that got comms workings again. > > The reason I am writing is because of wunderfixer behavior that I though I > had fully got my head around last week, but apparently not. :-/ > > My "outage" this morning started at 9:17 a.m. EDT., showing here: > > May 4 09:15:15 nixie weewx[9809]: manager: Added record 2019-05-04 > 09:15:00 EDT (1556975700) to database 'weewx.sdb' > May 4 09:15:15 nixie weewx[9809]: manager: Added record 2019-05-04 > 09:15:00 EDT (1556975700) to daily summary in 'weewx.sdb' > May 4 09:16:15 nixie weewx[9809]: manager: Added record 2019-05-04 > 09:16:00 EDT (1556975760) to database 'weewx.sdb' > May 4 09:16:15 nixie weewx[9809]: manager: Added record 2019-05-04 > 09:16:00 EDT (1556975760) to daily summary in 'weewx.sdb' > May 4 09:17:01 nixie CRON[21186]: (root) CMD ( cd / && run-parts > --report /etc/cron.hourly) > May 4 09:17:29 nixie weewx[9809]: wmr300: Loop data packets in heartbeat > interval = 0 > May 4 09:17:29 nixie weewx[9809]: wmr300: No loop data in heartbeat > interval, restarting > May 4 09:17:50 nixie weewx[9809]: wmr300: Loop data packets in heartbeat > interval = 0 > May 4 09:17:50 nixie weewx[9809]: wmr300: No loop data in heartbeat > interval, restarting > May 4 09:18:12 nixie weewx[9809]: wmr300: Loop data packets in heartbeat > interval = 0 > May 4 09:18:12 nixie weewx[9809]: wmr300: No loop data in heartbeat > interval, restarting > ... > > And the next recorded record after watchdog remediation was at 9:43 a.m. > > May 4 09:43:37 nixie weewx[382]: manager: Added record 2019-05-04 > 09:17:00 EDT (1556975820) to database 'weewx.sdb' > > *So WU would have been missing any data between 9:16 a.m. and 9:44 a.m...* > > In my weewx_watchdlog logs, which run wunderfixer for just such an > occasion, I see that wunderfixer kicked in to upload missing records, which > is expected, except *look closely!* > > Wunderfixer kept trying to upload records for every 1-minute of archive > data, but from what I learned last week, I expected it would only try to > upload records on roughly 5-minute boundaries, which I am further fudging > with --epsilon=125 to workaround the WU issue with fractional minutes not > aligned exactly to the 5-minute buckets. > > Scroll past for more... > > Sat 4 May 09:52:11 EDT 2019 Using configuration file > /etc/weewx/weewx.conf. > Sat 4 May 09:52:11 EDT 2019 Using database binding 'wx_binding', which is > bound to database 'archive_sqlite' > Sat 4 May 09:52:11 EDT 2019 Weather Underground Station: KMIDEARB5 > Sat 4 May 09:52:11 EDT 2019 Date to check: 2019-05-04 > Sat 4 May 09:52:11 EDT 2019 Number of archive records: 592 > Sat 4 May 09:52:11 EDT 2019 Number of WU records: 115 > Sat 4 May 09:52:11 EDT 2019 Number of missing records: 23 > Sat 4 May 09:52:11 EDT 2019 > Sat 4 May 09:52:11 EDT 2019 Missing records: > Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:20:00 EDT (1556976000); > 29.282"; 49.6F; 75%; 1.8 mph; 340 deg; 0.0 mph gust; 42.0F; 0.00" rain > ...published. > Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:21:00 EDT (1556976060); > 29.282"; 49.6F; 75%; 2.5 mph; 330 deg; 1.8 mph gust; 42.0F; 0.00" rain > ...published. > Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:22:00 EDT (1556976120); > 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 2.9 mph gust; 42.2F; 0.00" rain > ...published. > Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:23:00 EDT (1556976180); > 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 0.0 mph gust; 42.2F; 0.00" rain > ...published. > Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:24:00 EDT (1556976240); > 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 4.0 mph gust; 42.2F; 0.00" rai
[weewx-user] Interesting wunderfixer behavior
Hey, WeeWX'ers! TL:DR, scroll to the bottom for the question and scroll back as you need. ;-) I posted last week about WU "capacity" issues and the need to increase wunderfixer timeout from 10 seconds to 20 seconds to compensate. That's working great, so far. This morning my RPI -> WMR300 comms went south due to the well-known, but as of yet unfixed issue somewhere between the Kernel/USB handling and the flakey firmware on the WMR300. :-/ My weewx_watchdog script went through all the paces that I designed it to do and all there worked perfectly. After my watchdog tried the weewx restart (which wasn't enough [never is in my case]), then after another 10 minutes passed, it did the reboot, and that got comms workings again. The reason I am writing is because of wunderfixer behavior that I though I had fully got my head around last week, but apparently not. :-/ My "outage" this morning started at 9:17 a.m. EDT., showing here: May 4 09:15:15 nixie weewx[9809]: manager: Added record 2019-05-04 09:15:00 EDT (1556975700) to database 'weewx.sdb' May 4 09:15:15 nixie weewx[9809]: manager: Added record 2019-05-04 09:15:00 EDT (1556975700) to daily summary in 'weewx.sdb' May 4 09:16:15 nixie weewx[9809]: manager: Added record 2019-05-04 09:16:00 EDT (1556975760) to database 'weewx.sdb' May 4 09:16:15 nixie weewx[9809]: manager: Added record 2019-05-04 09:16:00 EDT (1556975760) to daily summary in 'weewx.sdb' May 4 09:17:01 nixie CRON[21186]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) May 4 09:17:29 nixie weewx[9809]: wmr300: Loop data packets in heartbeat interval = 0 May 4 09:17:29 nixie weewx[9809]: wmr300: No loop data in heartbeat interval, restarting May 4 09:17:50 nixie weewx[9809]: wmr300: Loop data packets in heartbeat interval = 0 May 4 09:17:50 nixie weewx[9809]: wmr300: No loop data in heartbeat interval, restarting May 4 09:18:12 nixie weewx[9809]: wmr300: Loop data packets in heartbeat interval = 0 May 4 09:18:12 nixie weewx[9809]: wmr300: No loop data in heartbeat interval, restarting ... And the next recorded record after watchdog remediation was at 9:43 a.m. May 4 09:43:37 nixie weewx[382]: manager: Added record 2019-05-04 09:17:00 EDT (1556975820) to database 'weewx.sdb' So WU would have been missing any data between 9:16 a.m. and 9:44 a.m... In my weewx_watchdlog logs, which run wunderfixer for just such an occasion, I see that wunderfixer kicked in to upload missing records, which is expected, except look closely! Wunderfixer kept trying to upload records for every 1-minute of archive data, but from what I learned last week, I expected it would only try to upload records on roughly 5-minute boundaries, which I am further fudging with --epsilon=125 to workaround the WU issue with fractional minutes not aligned exactly to the 5-minute buckets. Scroll past for more... Sat 4 May 09:52:11 EDT 2019 Using configuration file /etc/weewx/weewx.conf. Sat 4 May 09:52:11 EDT 2019 Using database binding 'wx_binding', which is bound to database 'archive_sqlite' Sat 4 May 09:52:11 EDT 2019 Weather Underground Station: KMIDEARB5 Sat 4 May 09:52:11 EDT 2019 Date to check: 2019-05-04 Sat 4 May 09:52:11 EDT 2019 Number of archive records: 592 Sat 4 May 09:52:11 EDT 2019 Number of WU records: 115 Sat 4 May 09:52:11 EDT 2019 Number of missing records: 23 Sat 4 May 09:52:11 EDT 2019 Sat 4 May 09:52:11 EDT 2019 Missing records: Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:20:00 EDT (1556976000); 29.282"; 49.6F; 75%; 1.8 mph; 340 deg; 0.0 mph gust; 42.0F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:21:00 EDT (1556976060); 29.282"; 49.6F; 75%; 2.5 mph; 330 deg; 1.8 mph gust; 42.0F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:22:00 EDT (1556976120); 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 2.9 mph gust; 42.2F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:23:00 EDT (1556976180); 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 0.0 mph gust; 42.2F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:24:00 EDT (1556976240); 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 4.0 mph gust; 42.2F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:25:00 EDT (1556976300); 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 0.0 mph gust; 42.2F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:26:00 EDT (1556976360); 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 4.0 mph gust; 42.2F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:27:00 EDT (1556976420); 29.282"; 49.8F; 75%; 2.5 mph; 330 deg; 0.0 mph gust; 42.2F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:28:00 EDT (1556976480); 29.282"; 50.0F; 75%; 2.5 mph; 330 deg; 1.3 mph gust; 42.4F; 0.00" rain ...published. Sat 4 May 09:52:11 EDT 2019 2019-05-04 09:29:00 EDT (1556976540); 29.282"; 50.0F; 76%;
