The best way to have wee_import calculate interval is really dependent on the underlying WU data and at times the reliability of WU. There are three ways that wee_import calculates interval; firstly by the simple difference in timestamps between successive records, by use of a fixed number defined in the import config file or by use of the archive_interval setting in weewx.conf. If your WU data is patchy or WU is having a bad day (at times I have seen the WU API return a different set of results each time you query the same historical day) deriving interval from successive records may lead to many incorrect interval values. How bad the discrepancy will be and how often it occurs will depend on your data. In such cases if you know the data was posted every x minutes you can use value x in the import config file to set a fixed interval or if x happens to be the same as your archive_interval setting in weewx.conf you can use that value. This is covered in the WU interval <http://weewx.com/docs/utilities.htm#wu_interval57> section in the Utilities Guide <http://weewx.com/docs/utilities.htm>.
Gary On Saturday, 22 February 2020 17:06:29 UTC+10, Nicolas CUVILLIER wrote: > > Thanks Gary > > On the test on current_rain, my main goal was more to not have wee_import > to hangs... > > I will create a new post on interval, because the values returned by WU > are quite wrong... > But I need to load all my historical data : June to 2015 to Nov 2017 is > done. > My PI is quite busy running days and nights :-) > > I will pull service request in future > Nicolas > > Le samedi 22 février 2020 07:30:10 UTC+4, gjr80 a écrit : >> >> Below issues should be fixed now. The epoch issue was interesting. What I >> neglected on my post in weewx-user was that according to WU's new API >> documentation the epoch field in their station history API response is in >> seconds since epoch, given the OverflowError encountered it would appear >> that is not what they are returning. But given WUs recent track record that >> does not surprise me. >> >> Gary >> >> On Saturday, 22 February 2020 09:46:55 UTC+10, gjr80 wrote: >>> >>> Nicolas, >>> >>> Thanks for the feedback, I will get onto looking at each issue. Some >>> initial comments below. >>> >>> Gary >>> >>> On Friday, 21 February 2020 18:54:29 UTC+10, Nicolas CUVILLIER wrote: >>>> >>>> Gary >>>> >>>> I am trying to import my historical data from underground. >>>> If I understood you wrote the utility >>>> >>>> I got some problem, and here is how I was able to work around the three >>>> issues I encountered: >>>> Note that this are my first lines in Python, I'm more a node's guy, >>>> then double check my Python proposals... >>>> >>>> >>>> in wee_import.py (v4.00.b12), line 1068, current_rain can be None, and >>>> the utility fail, and exit, please can you change the test: >>>> if current_rain >= last_rain: >>>> into >>>> if current_rain is not None and current_rain >= last_rain: >>>> >>>> Probably do need to catch current_rain == None, it really should not >>> occur but if WU was to respond with an invalid rain value under some >>> circumstances wee_import could take this as the value None. >>> >>> >>>> in wee_import, line 795, the test fail, I had to remove this test: >>>> *WeeWX 4.0.0 or greater is required, found 4.0.0b12. Nothing done, >>>> exiting* >>>> >>>> Was set to 4.0.0 in anticipation of 4.0 release, will set it to an >>> earlier beta. >>> >>> >>>> >>>> in wuimport.py, line 297, can you replace the test, which is failing >>>> sometime with an exit: >>>> (the expection does not initialise _date, not sure it is what is wanted >>>> ?) >>>> if obs == 'epoch': >>>> try: >>>> _date = datetime.date.fromtimestamp(_flat_record['epoch']) >>>> except ValueError: >>>> _flat_record['epoch'] = _flat_record['epoch'] // 1000 >>>> by >>>> if _flat_record['epoch'] > 1000000000000: >>>> _flat_record['epoch'] = _flat_record['epoch'] // 1000 >>>> >>>> # do we need to use a try here ? >>>> # what to put in _date if the function fail ? >>>> _date = datetime.date.fromtimestamp(_flat_record['epoch']) >>>> >>>> I need to look back at the code again but I believe this was handle the >>> case where the timestamp received was in milliseconds (a la java) rather >>> than seconds since epoch. _date is never used, rather we are looking >>> for the exception that would be thrown if the timestamp was in >>> milliseconds. Looking back at your weewx user thread it appears I need to >>> trap OverflowError as well, odd as I distinctly remember testing that >>> portion of the code. At this stage my preference is for the try..except >>> as it seems more pythonic to me. Would need to better comment the code >>> though. Like I said, need to look at the code. >>> >>> I am running the import for now, but it will take some time. >>>> I'll send you more comments if I get other problem. >>>> thanks >>>> Nicolas >>>> NB: Is it possible to directly push change proposals in GitHub ? >>>> >>> >>> Best way to submits changes is via GitHub pull request, though this does >>> require you to fork the WeeWX repo and submit the pull request via your >>> fork. >>> >>> >> -- You received this message because you are subscribed to the Google Groups "weewx-development" group. To unsubscribe from this group and stop receiving emails from it, send an email to weewx-development+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-development/ebc8b011-429f-48d2-bf48-17c22ed1c867%40googlegroups.com.
